JP2007067694A - Apparatus and method for encoding image, camera, and portable terminal equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding apparatus capable of correcting the blur of an animation generated by a camera shake in photographing by detecting the movement of the image and segmenting the image according to the movement, and performing recording and transmitting; and to provide an image encoding method. <P>SOLUTION: The image encoding apparatus includes a first encoding means 1 for encoding an input image, and outputting a motion vector which is used for encoding; a segmenting position determining means 2 for determining the segmenting position of the image with the use of the motion vector which is outputted from the first encoding means; an image segmenting means 3 for segmenting the input image in a prescribed size according to the image segmenting position which is outputted from the segmenting position determining means 2; and a second encoding means for encoding the segmented image which is outputted from the image segmenting means 3. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding technique for correcting and recording / transmitting moving image blur caused by camera shake during shooting.

  2. Description of the Related Art Conventionally, moving image recording or moving image transmission in a digital camera, a mobile terminal device with a camera, or the like has become a problem because camera shake tends to occur as the mobile terminal becomes smaller and lighter. As a method for correcting camera shake of a moving image, there are a method of providing a mechanical mechanism for correcting vibration of an optical system, a method of detecting a motion from an image, and a circuit for cutting out the image in accordance with the motion.

  Also, in motion picture digital recording and digital transmission, motion compensated predictive coding such as MPEG-4 and MPEG-2 is often employed to reduce the amount of data. When motion-compensated predictive coding is performed, when a blurred image is input, there is a problem that the image is not only disturbed and unsightly, but also the coding efficiency is lowered. In the case of such an encoding apparatus, a motion vector is obtained for each block in order to perform motion compensation prediction, so that a circuit is shared between motion vector detection for encoding and motion detection for camera shake correction. Trials have also been proposed (for example, Patent Document 1).

According to such a conventional technique, an input image is divided into blocks, a motion vector of each block is obtained, a camera shake vector is detected from each motion vector, the input image is corrected by the camera shake vector, and the corrected image is encoded. Turn into. At this time, the motion vector of each block is corrected with the hand movement vector, and motion compensation predictive coding is performed with the corrected motion vector.
JP-A-9-130748 (page 17 and FIG. 17)

  According to such a conventional technique, an input image is cut out according to a camera shake vector, and the cut out image is encoded as a corrected image. When the corrected image is encoded, block division is performed by dividing the input image when obtaining the camera shake vector. Therefore, there is a problem that even if the motion vector of each block is corrected by a hand movement vector, the motion vector is not necessarily the optimum motion vector for encoding.

  The present invention has been made to solve the above-described problems, and by performing the encoding twice, the motion vector for camera shake correction is detected and the camera shake is corrected without increasing the circuit scale. It is an object of the present invention to obtain an encoding device that can detect a motion vector suitable for encoding and efficiently compress an image.

The second aspect of the present invention also provides an encoding method in which a motion vector for camera shake correction is detected using an existing encoder by performing encoding twice, and the image is compressed by correcting the camera shake. The purpose.

  The present invention has been made to solve the above-described problems, and detects a first motion vector based on an input image and a previous image which is an image one frame before the input image, and performs a first prediction. A first image predicting unit for obtaining an image; a cutout position determining unit for determining an image cutout position based on the first motion vector; and the input image based on the image cutout position output from the cutout position determining unit. And a second image that obtains a second predicted image by detecting a second motion vector based on the clipped image corresponding to the clipped image and the previous image. Image coding apparatus including prediction means, and second variable length coding means for coding the output of the second image prediction means, and an image coding method corresponding thereto Is to provide.

  According to the present invention, by performing encoding twice, a motion vector for camera shake correction is detected in the first encoding, and an image is cut out according to the motion vector, and encoding is performed in the second encoding. By detecting and encoding a motion vector suitable for the image, an image encoding apparatus and an image encoding method capable of correcting camera shake of an input image and efficiently compressing an image are obtained.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

Embodiment 1
Embodiment 1 of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of an image coding apparatus according to Embodiment 1 of the present invention. In the figure, 1 is a first encoding means for encoding an input image, 2 is a cutout position determining means for determining an image cutout position based on a motion vector output from the first encoding means 1, Reference numeral 3 denotes an image cutout unit that cuts out an input image according to the image cutout position output from the cutout position determination unit 2, and 4 denotes a second encoding unit that encodes the cutout image output from the image cutout unit 3.

  Next, the operation will be described. The first and second encoding units are encoding units that detect the motion of the input image as a motion vector and perform motion compensation prediction encoding using the detected motion vector. First, the input image is input to the first encoding means 1 and subjected to motion compensation prediction encoding, and a motion vector used for encoding is output. For example, when the input image is divided into blocks of 4 horizontal divisions and 3 vertical divisions, motion vectors of 12 blocks as shown in FIG. 2 are output. A configuration example of the first encoding means 1 will be described later in detail.

  Next, the cut-out position determination unit 2 determines a camera shake vector indicating the motion of the entire screen using the motion vector. For example, in the case of FIG. 2, assuming that the most frequently occurring vector among the 12 output motion vectors is the camera shake vector, the camera shake vector is as shown in FIG. Note that the method for determining the hand movement vector may be an average of all motion vectors, or may be an intermediate value of each distribution of the horizontal and vertical components of the motion vector. Alternatively, a method may be used in which a histogram is obtained for each of the horizontal and vertical components of a motion vector, and an average is obtained by excluding a value that is more than a standard deviation from each average.

  The cutout position determination unit 2 determines the image cutout position of the current frame from the camera shake vector and the image cutout position of the previous frame. For example, in FIG. 4, 11 is the previous image that is the input image of the previous frame, 12 is the image cut-out position of the previous frame when the image position is represented by the upper left corner of the image, and 13 is the image after cutting the previous frame. Is an input image of the current frame, 15 is a cut-out image of the previous frame, and 16 is a camera shake vector. In general, in motion-compensated predictive coding such as MPEG-2 and MPEG-4, the motion vector indicates the block position of the previous frame corresponding to the block of the current frame. A vector 17 with the end points replaced is a vector indicating the movement from the previous frame to the current frame. Therefore, a position obtained by moving the image cutout position 12 of the previous frame by the vector 17 becomes an image cutout position 18 of the current frame. That is, 19 in FIG. 4 is an image after cutting out the current frame. Note that the image cutout position 18 needs to be restricted so that the cutout image 19 does not go out of the screen.

  In accordance with the image cutout position determined by the cutout position determination unit 2, the image cutout unit 3 cuts out the input image and inputs the image 19 after cutout to the second encoding unit 4. The second encoding means 4 encodes the cut image and outputs encoded data.

  Next, a configuration example of the first and second encoding means will be described. FIG. 5 shows a configuration example of the second encoding means, and FIG. 6 shows a configuration example of the first encoding means. In FIG. 5, the input image after clipping is given to the first input of the difference unit 101, the first input of the switch 102, and the first input of the motion compensation prediction unit 111. A predicted image output from the motion compensation prediction unit 111 is given to the second input of the difference unit 101, and an output of the difference unit 101 is given to the second input of the switch 102. The third input of the switch 102 is given the intra / inter determination result output from the motion compensation prediction means 111, and the output of the switch 102 is input to the DCT means 103. The output of the DCT unit 103 is input to the quantization unit 104, and the output of the quantization unit 104 is supplied to the first input of the variable length encoding unit 105 and the inverse quantization unit 106. The output of the inverse quantization means 106 is input to the inverse DCT means 107, and the output of the inverse DCT means 107 is given to the first input of the adder 108 and the first input of the switch 109. The second input of the adder 108 is given a predicted image output from the motion compensation prediction unit 111. The output of the adder 108 is given to the second input of the switch 109, and the intra / inter determination result outputted from the motion compensation prediction means 111 is given to the third input of the switch 109. The output of the switch 109 is input to the memory 110. The output of the memory 110 is given to the second input of the motion compensation prediction means 111. The motion vector output from the motion compensation prediction unit 111 is given to the second input of the variable length encoding unit 105, the output of the variable length encoding unit 105 is input to the buffer 112, and the encoded data is output from the buffer 112. Is done.

  Next, the operation of the second encoding means will be specifically described based on FIG. The difference between the cut-out image output from the image cut-out unit 3 and the predicted image output from the motion compensation prediction unit 111 described later is obtained by the differentiator 101. Based on the intra / inter determination result output from the motion compensation prediction unit 111 described later, the switching unit 102 inputs the cut-out image input in the case of intra to the DCT unit 103 as it is, and in the case of inter the difference unit The difference from the predicted image obtained in 101 is input to the DCT means 103. The DCT unit 103 performs DCT conversion for each block on the difference between the input image after cutout or the predicted image and outputs a DCT coefficient. The DCT coefficient is quantized by the quantization means 104. The quantized DCT coefficients are encoded by the variable length encoding means 105, dequantized by the inverse quantization means 106, and inverse DCT transformed by the inverse DCT means 107. In the case of intra, since the inverse DCT transformed data is a decoded image, it is stored in the memory 110 as it is. In the case of inter, a decoded image is created by adding the inverse DCT-transformed data to the predicted image and stored in the memory 110. Further, the data encoded by the variable length encoding means 105 is output to the outside via the buffer 112. In the second encoding means, the portion excluding the variable length encoding means 105 and the buffer 112 constitutes an image prediction means 113.

  The motion compensation prediction unit 111 divides the input cut-out image into blocks composed of a plurality of pixels, and compares them with past decoded images stored in the memory 110, thereby obtaining a motion vector for each block. As a method for obtaining the motion vector, for example, block matching is used. That is, the difference between each block of the input clipped image and the block at an arbitrary position of the past decoded image is obtained, and the block position (relative position) of the past decoded image having the smallest sum of absolute differences. Is determined as a motion vector. The motion compensation prediction unit 111 further creates a predicted image according to the obtained motion vector. The motion compensation prediction unit 111 calculates an evaluation value of a prediction error according to the difference between the clipped image input for each block and the prediction image. If the prediction error is larger than a predetermined value, the motion compensation prediction Intra / inter determination is performed as intra which is an intra-screen mode in which motion compensation is not performed, and as inter which is a motion compensation prediction mode in which motion compensation prediction is performed otherwise. The evaluation value of the prediction error may be a sum of absolute differences obtained by block matching, or a sum of squared differences between the predicted image and the input image after extraction. The intra / inter determination is not limited to this. For example, the variance for each block of the input cut-out image may be obtained, and if the variance is small, it may be determined as intra. In this way, the motion compensation prediction unit 111 outputs the intra / inter determination result, the predicted image, and the motion vector. In addition, when the intra / inter determination result is intra, it is good also as a structure which does not output a prediction image and a motion vector.

  The variable length encoding unit 105 encodes the quantized DCT coefficient output from the quantization unit 104 and the motion vector output from the motion compensation prediction unit 111, and outputs the encoded data to the buffer 112. The buffer 112 outputs the encoded data to the outside.

  Next, the operation of the first encoding means will be described with reference to FIG. 6, the same reference numerals as those in FIG. 5 denote the same or corresponding parts. The configuration of the first encoding unit is basically the same as the configuration of the second encoding unit, but the motion vector output from the motion compensation prediction unit 111 is not only the variable length encoding unit 105, The difference is that the data is output to the cut-out position determination unit 2 and the encoded data is not output from the buffer 112 to the outside.

  In the first encoding means, since it is not necessary to output encoded data from the buffer 112, the size of the buffer 112 is reduced, and the encoded data output from the variable length encoding means 105 is controlled to always be overwritten. May be. Further, the variable length encoding means 105 and the buffer 112 need not be provided. That is, the image prediction unit 113 may be configured only. In this case, the output from the quantization means 104 is given only to the inverse quantization means 106, and the output from the motion compensation prediction means 111 is outputted only to the cutout position determination means 2.

  As described above, the first and second encoding units have substantially the same configuration. Therefore, when the same circuit is shared and used as the first encoding unit, the motion output from the motion compensation prediction unit 111 is used. When the vector is controlled so that it can be output separately from the encoded data and used as the second encoding means, the encoded data can be output from the buffer 112 without increasing the circuit scale. Thus, an image encoding device having a camera shake correction function can be configured. When the same circuit is used as the first and second encoding means, the memory 110 cannot be shared and two memories need to be provided. Alternatively, a memory having a size capable of recording two images may be provided, and selection means for selecting and outputting the two recorded images according to the first and second encodings may be provided.

  As described above, the image coding apparatus according to Embodiment 1 corrects camera shake of an input image using a camera shake vector obtained from a motion vector, and encodes a camera shake-corrected image. It can be used to efficiently compress images.

Embodiment 2
FIG. 7 shows an image encoding apparatus according to Embodiment 2, and FIG. 8 shows a flowchart showing an image encoding method by this image encoding apparatus. In FIG. 7, 201 is a CPU, 202 is an encoder, and 203 is a memory. These CPU 201, encoder 202, and memory 203 are connected by a CPU bus 204. Here, the encoder 202 includes the encoding unit 1 (or 4), the cutout position determining unit 2 and the image cutout unit 3 in the first embodiment. The input image is input to the memory 203 from the outside, the encoder 202 is activated under the control of the CPU 201, the input image in the memory 203 is encoded, and the encoded data is stored in the memory 203. The encoder 202 is configured so that the CPU 201 can control parameter settings such as the image size to be encoded, output ON / OFF of the motion vector used for encoding, and output ON / OFF of the intra / inter determination result. The Here, information such as a motion vector used for encoding and an intra / inter determination result is stored in an area (not shown) different from the encoded data on the memory 203.

  Next, the image coding method according to the second embodiment will be described with reference to FIG. First, the CPU 201 sets operation parameters of the encoder 202. The image size to be encoded is set to the image size of the input image, and the output of the motion vector to the clip position determination unit 2 is turned ON. Then, the encoder is activated (301). An image cutout position is determined from the motion vector output here (302). As a method for determining the image cutout position, for example, as described in the first embodiment, the average of the motion vectors of the respective blocks is set as a shake vector, and the image cutout position is determined according to the shake vector. Next, an image is cut out with a predetermined image size in accordance with the determined cut-out position (303). Finally, the CPU 201 sets the image size to be encoded by the encoder 202 to the image size after clipping, and activates the encoder 202 to encode the image after clipping (304). If the next frame is to be encoded, the process returns to step 301. If the encoding is to be ended, the process is ended. A series of encoded data output from step 304 becomes image encoded data with camera shake correction.

  As described above, according to the second embodiment, it is possible to obtain an image encoding method / image encoding apparatus having a camera shake correction effect by activating an existing encoder twice.

  In the second embodiment, when the first encoder setting to start-up (301) is performed, the bit rate setting to the encoder 202 is increased in order to prevent the motion vector detection accuracy from being reduced due to coding distortion. Thus, it is possible to obtain camera shake correction with few erroneous determinations. For example, when the desired bit rate is 64 kbit / s, the bit rate setting to the encoder 202 is 64 kbit / s in the second encoder activation (304), but in the first encoder setting to activation (301). The bit rate setting for the encoder 202 is, for example, 256 kbit / s.

  Alternatively, when the encoder 202 includes the encoding unit configured as shown in FIG. 6 described in the first embodiment, the quantization step width in the quantization unit 104 is set in the first encoder setting to activation (301). It may be controlled to be fixed at a minimum value, for example, 1 so that encoding distortion does not occur or becomes sufficiently small.

  Alternatively, if the encoder 202 is configured to increase the processing to search for motion vectors when the bit rate is set to a low bit rate, quantization is performed in the first encoder setting to activation (301). If the quantization step width in the means 104 can be set to a large value, for example, an integer from 1 to 31, for example, it is fixed to 20, for example, the processing applied to the motion vector search is increased, and the motion vector accuracy is controlled to be high. May be.

  As described above, in the first encoder setting to start-up (301), by setting a high bit rate or optimizing the quantization step according to the apparatus, a motion vector that differs greatly from the original motion due to coding distortion is obtained. Detection can be prevented, and camera shake correction with high accuracy is possible.

  Further, in the second embodiment, the encoder 202 has the coding unit configured as shown in FIG. 6 described in the first embodiment and does not output a motion vector when the intra / inter determination result is intra. In such a configuration, in the first encoder activation (301), if control is performed so that the intra / inter determination result of the motion compensation prediction means 111 is always inter, motion vectors are obtained for all blocks. Therefore, it is possible to obtain an image encoding device having a highly accurate camera shake correction function. For example, when the intra / inter determination is configured to determine intra when the prediction error is larger than a predetermined value, in the first encoder setting to activation (301), this predetermined value can take the maximum possible prediction error. By setting the value, it is possible to control so that the intra / inter determination result is always inter, and it is possible to detect camera shake vectors for all blocks.

  Furthermore, in Embodiment 1 and Embodiment 2 described above, the motion vector of each block is detected in the first encoding, and the image cutout position is determined based on this motion vector. When the encoding means is an encoding method using a global motion vector indicating the motion of the entire screen, the first encoding outputs the global motion vector and determines the image cutout position based on the global motion vector. You may comprise.

Embodiment 3
In Embodiment 1 and Embodiment 2 described above, the motion vector is detected in the first encoding, the image cutout position is determined based on this motion vector, and the image is cut out and encoded. In the second encoding, a motion vector and an intra / inter determination result may be output, and an image cutout position may be determined based on the motion vector and the intra / inter determination result of each block.

  FIG. 9 is a block diagram showing an image coding apparatus according to the third embodiment as described above. In the figure, reference numeral 1a denotes a first encoding unit that encodes an input image, and a motion vector output from the first encoding unit and an intra / inter determination result are input to the cut-out position determination unit 2a. In accordance with the image cutout position output from the cutout position determining means 2a, the image cutout means 3 cuts out the input image and inputs it to the second encoding means 4. The second encoding means 4 encodes the cut image and outputs encoded data.

  Next, the operation will be described. The first encoding unit 1a is configured as shown in FIG. 6 described in the first embodiment, for example. The difference from the first encoding unit 1 of the first embodiment is not only the motion vector obtained by the motion compensation prediction unit 111 but also the intra / inter determination result (mode information indicating whether the motion compensation prediction mode or the in-screen mode). ) Is also input to the cut-out position determining means 2a.

  The cutout position determination unit 2a determines the image cutout position based on the motion vector output from the first encoding unit 1 and the intra / inter determination result. For example, for a block determined to be intra, the motion vector is set to (0, 0), the average value of the motion vectors of the entire screen is obtained, and this is used as the camera shake vector. Based on this camera shake vector, the image cutout position is determined as shown in FIG.

  Since the operations of the image cutout unit 3 and the second encoding unit 4 are the same as those in the first embodiment, description thereof is omitted.

  In Embodiment 3, the cutout position determination unit 2a determines the image cutout position with the motion vector set to (0, 0) for the block determined to be intra, but the block determined to be inter The average value of only the motion vectors may be used as a camera shake vector, and the image cutout position may be determined based on the camera shake vector. Further, the method of calculating the hand movement vector is not limited to the average, and a method of selecting an intermediate value or a vector having the highest frequency may be used.

  By configuring as described above, it is possible to improve the accuracy of camera shake correction by handling that there is no motion in the in-screen mode in which the reliability of the motion vector is low.

  In the first to third embodiments, the first encoding unit is configured as shown in FIG. 6, and the motion vector or intra / inter determination result obtained by the motion compensation prediction unit 111 is sent to the cut-out position determining unit 2. However, the configuration of the first encoding unit is not limited to this. For example, the first encoding unit has the same configuration as the second encoding unit shown in FIG. It is possible to obtain a motion vector or an intra / inter determination result by decoding.

Embodiment 4
FIG. 10 is a block diagram showing an image coding apparatus according to the fourth embodiment. In the figure, 1b is a first encoding means for encoding an input image, and the first encoding means 1b extracts the motion vector and the intra / inter determination result to the position determining means 2a, and the code amount control information to the first. 2 to the encoding means 4a. The image cutout unit 3 cuts out an input image according to the image cutout position output from the cutout position determination unit 2a, and inputs the input image to the second encoding unit 4a. The second encoding unit 4a encodes the cut-out image according to the code amount control information output from the first encoding unit 1b, and outputs encoded data.

  Next, the operation will be described. The first encoding means 1b is configured as shown in FIG. 6 described in the first embodiment, for example. Here, the first encoding unit 1 of the first embodiment is different from the first embodiment in that not only the motion vector determined by the motion compensation prediction unit 111 but also an intra / inter determination result is output, and variable length encoding. The code amount control information calculated based on the code amount output from the means 105 is output.

  The motion vector and the intra / inter determination result output from the first encoding unit 1b are input to the cutout position determination unit 2a to determine the image cutout position, and the image cutout unit 3 further determines the image cutout position according to the image cutout position. The input image is cut out. Since the operations of the cutout position determination unit 2a and the image cutout unit 3 are the same as those in the third embodiment, description thereof is omitted.

  The second encoding unit 4a encodes the cut-out image output from the image cut-out unit 3. At this time, the code amount is controlled according to the code amount control information output from the first encoding means 1b. For example, the first encoding unit 1b is configured to control the quantization unit 104 to perform quantization with a fixed quantization step width and output the code amount for each frame as the code amount control information. At this time, since the code amount control information represents the complexity of the image for each frame, the second encoding unit 4a sets the target code amount of each frame according to this code amount control information, and controls the quantization step width. To do. For example, when the bit rate fluctuation for each frame is allowed and recording / transmission is performed with the highest possible image quality within the average bit rate, the target code amount of each frame is set to be proportional to the code amount control information, The quantization step width is controlled so that the generated code amount of the frame becomes equal to the target code amount. Here, as the code amount control information, for example, a code amount and a quantization step width, or a product of the code amount and the quantization step width is used. When the quantization unit 104 in the first encoding unit 1b uses a fixed quantization step width, the code amount itself may be used as the code amount control information.

  Alternatively, in the constant bit rate transmission, when the generated code amount for each frame is as constant as possible, by setting the initial value of the quantization step width of each frame by conversion inversely proportional to the code amount control information, It becomes easy to control the generated code amount of each frame to be constant.

  The first encoding unit 1b outputs the code amount for each block as code amount control information, and the second encoding unit determines the quantization parameter of each part in the frame according to the code amount control information. You may comprise.

  As described above, in the fourth embodiment, encoding is performed twice, so that not only camera shake correction but also code amount control for high image quality or constant bit rate can be performed. It is possible to obtain a high-quality image encoding device and an image encoding device capable of stable bit rate control while removing image disturbance.

It is a block diagram which shows the image coding apparatus in Embodiment 1 of this invention. It is explanatory drawing which shows the motion vector output from the 1st encoding means in Embodiment 1 of this invention. It is explanatory drawing which shows operation | movement of the cut-out position determination means in Embodiment 1 of this invention. It is explanatory drawing which shows operation | movement of the cut-out position determination means in Embodiment 1 of this invention. It is a block diagram which shows the example of 1 structure of the 2nd encoding means in Embodiment 1 of this invention. It is a block diagram which shows the example of 1 structure of the 1st encoding means in Embodiment 1 of this invention. It is a block diagram which shows the image coding apparatus in Embodiment 2 of this invention. It is a flowchart which shows the image coding method in Embodiment 2 of this invention. It is a block diagram which shows the image coding apparatus in Embodiment 3 of this invention. It is a block diagram which shows the image coding apparatus in Embodiment 4 of this invention.

Explanation of symbols

1, 1a, 1b first encoding means,
2, 2a Cutout position determining means,
3 image cropping means,
4, 4a second encoding means,
101 differentiator,
102 switch,
103 DCT means,
104 quantization means,
105 variable length encoding means,
106 inverse quantization means,
107 inverse DCT means,
108 adders,
109 switch,
110 memory,
111 motion compensation prediction means,
112 buffers,
113 Image predicting means 201 CPU,
202 encoder,
203 memory,
204 CPU bus.

Claims (19)

First image prediction means for detecting a first motion vector and obtaining a first predicted image based on an input image and a previous image that is an image one frame before the input image;
Cutout position determining means for determining an image cutout position based on the first motion vector;
Image cutout means for cutting out the input image into a cutout image of a predetermined image size based on the image cutout position output from the cutout position determining means;
Second image prediction means for detecting a second motion vector based on the clipped image and the clipped image corresponding to the previous image and obtaining a second predicted image;
An image encoding apparatus comprising: second variable length encoding means for encoding the output of the second image prediction means. The first image prediction means includes
Means for obtaining a first difference which is a difference between the input image and the first predicted image;
Means for orthogonally transforming the first difference or the input image and outputting a first transform coefficient;
Means for quantizing the first transform coefficient;
The second image prediction means includes:
Means for obtaining a second difference which is a difference between the clipped image and the second predicted image;
Means for orthogonally transforming the second difference or the cut-out image and outputting a second transform coefficient;
Means for quantizing the second transform coefficient,
2. The image encoding according to claim 1, wherein the first image predicting unit performs processing on the input image, and the second image predicting unit performs processing on a block unit obtained by dividing the cut image into a plurality of blocks. apparatus. First variable length encoding means for encoding the quantized first transform coefficient output from the first image prediction means;
3. The image according to claim 2, wherein a bit rate of encoded data by the first variable length encoding unit is set higher than a bit rate of encoded data by the second variable length encoding unit. Encoding device. 3. The image encoding apparatus according to claim 2, wherein the first image predicting means sets a quantization step width of the first transform coefficient to a fixed value. The first image prediction means blocks the motion compensation prediction mode and the in-screen mode based on the first difference magnitude, and the second image prediction means based on the second difference magnitude. Can be switched every time,
The first image prediction means outputs mode information indicating which of the motion compensation prediction mode and the in-screen mode has been switched to the cutout position determination means together with the first motion vector for each block. The image coding apparatus according to claim 2. The cutout position determination unit treats the motion vector of the block as (0, 0) when the mode information output from the first image prediction unit is an in-screen mode. Item 6. The image encoding device according to Item 5. The cutout position determination unit determines an image cutout position with reference to only a motion vector of a block whose mode information output from the first image prediction unit is a motion compensation prediction mode. 5. The image encoding device according to 5. The second image prediction means is set to switch between the motion compensation prediction mode and the in-screen mode for each block according to the magnitude of the second difference,
The image coding apparatus according to claim 2, wherein the first image predicting unit is set to always select a motion compensation prediction mode. 9. The method according to claim 1, wherein the first image prediction unit and the second image prediction unit are common image prediction units, and each necessary setting is set by an external setting unit. An image encoding apparatus according to claim 1. A camera comprising the image encoding device according to claim 1. A portable terminal device comprising the camera according to claim 10. A first image prediction step of detecting a first motion vector based on an input image and a previous image that is an image one frame before the input image to obtain a first predicted image;
A cutout position determining step for determining an image cutout position based on the first motion vector;
An image cutout step of cutting the input image into a cutout image of a predetermined image size based on the image cutout position output from the cutout position determination step;
A second image prediction step of detecting a second motion vector based on the clipped image and the clipped image corresponding to the previous image and obtaining a second predicted image;
An image encoding method comprising: a second variable length encoding step for encoding the output obtained in the second image prediction step. The first image prediction step includes:
Obtaining a first difference that is a difference between the input image and the first predicted image;
Orthogonally transforming the first difference or the input image and outputting a first transform coefficient;
Quantizing the first transform coefficient,
The second image prediction step includes:
Obtaining a second difference that is a difference between the clipped image and the second predicted image;
Orthogonally transforming the second difference or the clipped image and outputting a second transform coefficient;
Quantizing the second transform coefficient,
13. The image encoding according to claim 12, wherein the first image prediction step performs processing in units of blocks obtained by dividing the input image, and the second image prediction step includes dividing the cut-out image into a plurality of blocks. Method. A first variable length encoding step for encoding the quantized first transform coefficient output from the first image prediction step;
14. The image according to claim 13, wherein a bit rate of encoded data obtained by the first variable length encoding step is set higher than a bit rate of encoded data obtained by the second variable length encoding step. Encoding method. 14. The image encoding method according to claim 13, wherein in the first image prediction step, a quantization step width of the first transform coefficient is set to a fixed value. The first image prediction step blocks the motion compensation prediction mode and the in-screen mode based on the first difference size, and the second image prediction step based on the second difference size. Can be switched every time,
In the first image prediction step, mode information indicating which one of the motion compensation prediction mode and the in-screen mode is switched is output to the cutout position determination step together with the first motion vector for each block. The image encoding method according to claim 13. The cutout position determination step treats the motion vector of the block as (0, 0) when the mode information output from the first image prediction step is an in-screen mode. Item 17. The image encoding method according to Item 16. The cutout position determining step determines an image cutout position with reference to only a motion vector of a block whose mode information output from the first image prediction step is a motion compensation prediction mode. 16. The image encoding method according to 16. The second image prediction step is set to switch between the motion compensation prediction mode and the in-screen mode for each block according to the magnitude of the second difference.
The image encoding method according to claim 13, wherein the first image prediction step is set to always select a motion compensation prediction mode.

Images