JP2008193503A - Moving image encoding device and moving image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce throughput and to shorten a processing time by making image data to be processed small. <P>SOLUTION: This moving image encoding device encodes a difference between a predictive image generated by using a weighted reference image obtained by weighting a reference image and adding offset and an original image. Also there is provided a moving image encoding method. The moving image encoding device is provided with an image data reducing part for reducing data amounts of the original image and the reference image, a first weight estimating part for estimating a weight parameter on the basis of image data of the reduced original image and reference image, a weighted motion vector retrieving part for searching a motion vector on the basis of the weighted parameter, the original image and the reference image, a motion compensating part for using the motion vector and the reference image to generate a motion compensation image, and a second weight estimating part for estimating a weight parameter from the motion compensation image and the original image. The moving image encoding method has a configuration similar to a function of each part of the moving image encoding device. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image coding apparatus and a moving image coding method that perform moving image coding using inter-frame differences.

  In a moving image coding system that encodes a difference between a predicted image and an original image, a technique for applying a weight to the predicted image and adding an offset, and a technique for taking the difference between the original image and encoding the same are disclosed as patents. There is something as shown in Document 1.

In Patent Document 1, when the pixel value of the original image is X [i] and the pixel value of the predicted image is Y [i],
X [i]-(W * Y [i] + T)
Is encoded.

  If this value is close to 0, the compression ratio increases, which is a well-known fact if you have expertise in this field.

  Of course, it is important to set appropriate values of W and T in order to increase the compression rate.

  In the example described in Patent Document 1, W is obtained as in the following equation.

W = AVE [X [i]] / AVE [Y [i]]
When X [i] and Y [i] are positive correlations as shown in FIG. 4 and the distribution is along a straight line passing through the origin, W determined by equation (1) is , X [i] − (W * Y [i] + T) is effective for reducing the value. In this case, T = 0.

  Also, the motion compensated image mcref obtained by performing motion compensation using a standard image (reference image) that is not weighted using a motion vector is Y [i], and the encoding target image cur is X [ As i], W = AVE [X [i]] / AVE [Y [i]] is similarly obtained, whereby the weight can be obtained more accurately.

As a result, encoding can be effectively performed in the case of fading.
JP-T-2006-509467

  However, a problem arises when the distribution of x [i] and y [i] is not along a straight line passing through the origin. FIG. 5 shows an example of this.

  In the case of following the form of ax + b (b> 0), the value of y− (w * x + t) is small in the portion of x = (x_min + x_max) / 2, but in the vicinity of x = x_min or x = x_max The value of y− (w * x + t) increases and the code amount increases.

  That is, the problem is that the amount of code increases despite the time it takes to calculate the weight.

  The present invention has been made to solve the above-described problem, and encodes a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset to the original image. In the moving image encoding apparatus, an image data reduction unit that reduces the data amount of the original image and the reference image while retaining the features of the original image and the reference image, and an original image that has been reduced in data amount by the image data reduction unit, and A first weight estimator for estimating a weight parameter based on image data of a reference image; a weighted motion vector for searching for a motion vector based on the weight parameter estimated by the first weight estimator and the original image and the reference image; A search unit, a motion compensation unit that generates a motion compensated image using the motion vector searched by the weighted motion vector search unit and the reference image, and generated by the motion compensation unit Characterized in that the motion compensation image and the original image and a second weight estimation unit that estimates a weighting parameter.

  Thereby, when performing a motion vector search for obtaining a motion vector used in the motion compensation unit, a highly accurate motion vector search is performed by performing a weighted motion vector search. The weight parameter obtained by using the result becomes accurate.

  Furthermore, as another configuration, the least square method is used for weight estimation. Thereby, the accuracy of weight estimation can be improved and the weight parameter can be obtained accurately.

  The moving picture coding method has the same configuration as the function of each unit of the moving picture coding apparatus.

  According to this moving image encoding method, the data amount of the original image and the reference image is reduced and processing such as weight estimation is performed as in the case of the moving image encoding apparatus.

  As an effect of the first solving means, since the weight parameter is accurately obtained, the difference between the weighted motion compensated image and the original image becomes small, and the encoding result of the difference becomes small. Furthermore, since the data amount of the original image and the reference image is reduced and processing such as weight estimation is performed, the size of the data to be processed can be reduced, and the processing amount and the processing time can be reduced. Can do.

  As an effect of the second solving means, in addition to the effect of the first solving means, since the weight parameter is further accurately obtained, the difference between the weighted motion compensated image and the original image becomes small, The difference encoding result is reduced.

  Hereinafter, a moving picture coding apparatus and a moving picture coding method according to an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a functional block diagram showing a video encoding device for realizing the video encoding method according to the first embodiment, and FIG. 2 is a diagram for realizing the video encoding method according to the second embodiment. FIG. 3 is a functional block diagram showing a video encoding apparatus for realizing the video encoding method according to the third embodiment, and FIG. 7 is a fourth implementation diagram. FIG. 8 is a functional block diagram showing a video encoding device for realizing the video encoding method according to the embodiment, and FIG. 8 shows a video encoding device for realizing the video encoding method according to the fifth embodiment. It is a functional block diagram shown.

[First Embodiment]
A video encoding apparatus and video encoding method according to the first embodiment will be described with reference to FIG. The moving picture coding apparatus and the moving picture coding method according to the present embodiment finally outputs weights and offsets when an original image and a reference image are input.

  The moving image encoding device 1 includes a first thumbnail image generation unit 2, a second thumbnail image generation unit 3, a first weight estimation unit 4, a weighted motion vector search unit 5, a motion compensation unit 6, And a 2-weight estimation unit 7. Hereinafter, the configuration of each unit will be described in the order in which image data is input.

  From the original image input terminal 8, the original image to be compressed is input to the first thumbnail image generation unit 2. The first thumbnail image generation unit 2 is an image data reduction unit that reduces the data amount of the original image while leaving the features of the original image. After the original image input to the first thumbnail image generation unit 2 is reduced, from the first thumbnail image generation unit 2, the first weight estimation unit 4, the weighted motion vector search unit 5, the motion compensation unit 6, Each is sent to the second weight estimation unit 7 and processed in each unit. Also, the original image to be compressed is input from the original image input terminal 8 to the weighted motion vector search unit 5 and the second weight estimation unit 7 together with the first thumbnail image generation unit 2.

  On the other hand, a reference image used for compression is input from the reference image input terminal 9 to the second thumbnail image generation unit 3. The second thumbnail image generation unit 3 is an image data reduction unit that reduces the data amount of the reference image while leaving the characteristics of the reference image. The reference image input to the second thumbnail image generation unit 3 is sent from the second thumbnail image generation unit 3 to the first weight estimation unit 4, the weighted motion vector search unit 5, the motion compensation unit 6, and the second weight estimation unit 7. Are sent to and processed by each unit. Further, the reference image is input from the reference image input terminal 9 to the weighted motion vector search unit 5 and the motion compensation unit 6 together with the second thumbnail image generation unit 3.

  Hereinafter, specific processing in each unit will be described.

  The first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 have the following processing functions.

  The original image input from the original image input terminal 8 is sent to the first thumbnail image generation unit 2. The first thumbnail image generation unit 2 reduces the original image and sends it as a thumbnail original image to the first weight estimation unit 4.

  The reference image input from the reference image input terminal 9 is sent to the second thumbnail image generation unit 3. The second thumbnail image generation unit 3 reduces the reference image and sends it as a thumbnail reference image to the first weight estimation unit 4.

  The first weight estimation unit 4 has the following processing functions.

  The first weight estimation unit 4 receives the thumbnail original image from the first thumbnail image generation unit 2. Further, the thumbnail reference image is received from the second thumbnail image generation unit 3. The first weight estimation unit 4 estimates weight parameters (weight and offset) using the thumbnail original image and thumbnail reference image received from the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3. Specific processing will be described later. Next, the first weight estimation unit 4 sends the estimated weight and offset to the weighted motion vector search unit 5.

  The weighted motion vector search unit 5 has the following processing functions.

  The weighted motion vector search unit 5 searches the reference image for a motion vector of the original image. The weighted motion vector search unit 5 receives an original image from the original image input terminal 8 and receives a reference image from the reference image input terminal 9. Further, the first weight estimation unit 4 receives estimated values of weights and offsets. A motion vector is searched based on these data. The weighted motion vector search unit 5 outputs the motion vector obtained as a result of the search to the motion compensation unit 6.

  The motion compensation unit 6 has the following processing functions.

  The motion compensation unit 6 generates a motion compensated image using the reference image and the motion vector. The motion compensation unit 6 receives a reference image from the reference image input terminal 9 and receives a motion vector from the weighted motion vector search unit 5. The motion compensation unit 6 generates a motion compensation image from these reference images and motion vectors, and outputs the motion compensation image to the second weight estimation unit 7.

  The second weight estimation unit 7 has the following processing functions.

  The second weight estimation unit 7 receives the motion compensated image from the motion compensation unit 6 and receives the original image from the original image input terminal 8. Then, a weight and an offset are obtained from the motion compensated image and the original image. Specific processing will be described later. The second weight estimation unit 7 outputs the weight and offset to the weight and offset output terminal 10.

  Next, the moving picture coding method of the present invention will be described.

  First, a thumbnail image generation process is performed. This thumbnail image generation step is an image data reduction step for reducing the data amount of the original image and the reference image while leaving the features of the original image and the reference image. The image data is reduced in a state in which features such as movement on the screen and changes in brightness remain. Here, based on the original image and the reference image, the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 generate the thumbnail original image and the thumbnail reference image.

  Next, a first weight estimation step is performed. Here, the first weight estimation unit 4 roughly estimates the weight w0 and the offset t0 using the thumbnail original image and thumbnail reference image.

  Next, a weighted motion vector search process is performed. Here, the weight w 0 and the offset t 0 estimated by the first weight estimation unit 4, the original image from the original image input terminal 8, and the reference image from the reference image input terminal 9 are taken into the weighted motion vector search unit 5. The motion vector is searched using these weight w0 and offset t0, the original image, and the reference image. Then, the searched motion vector is output to the motion compensation unit 6.

  Next, a motion compensation process is performed. Here, the motion vector searched by the weighted motion vector search unit 5 and the reference image from the reference image input terminal 9 are taken into the motion compensation unit 6, and a motion compensated image is generated from the motion vector and the reference image. Then, the generated motion compensated image is output to the second weight estimation unit 7.

  Next, a second weight estimation step is performed. Here, the motion compensation image generated by the motion compensation unit 6 and the original image from the original image input terminal 8 are taken into the second weight estimation unit 7, and the second weight estimation unit is obtained from the motion compensation image and the original image. In step 7, the weight w1 and the offset t1 are obtained from the motion compensated image and the original image.

  Next, detailed operations in the respective units will be described below.

(First thumbnail image generation unit 2 and second thumbnail image generation unit 3)
The first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 perform a process of reducing the input image.

  By taking the sum for each m × m pixels of the input image and using this as one pixel of the reduced image, an image of 1 / m in length and width can be obtained. For example, if m = 4, an image of a quarter of the length and width is generated.

(First weight estimation unit 4)
The first weight estimation unit 4 performs the following processing.

x [i] is the i-th pixel value of the thumbnail reference image input from the second thumbnail image generation unit 3, and y [i] is the i-th pixel of the thumbnail original image input from the first thumbnail image generation unit 2. When the pixel value is given by the following formula, the relationship of equation (1) is considered.

  However, Σ indicates the total sum of the reduced image range.

A process for obtaining a weight w0 and an offset t0 that minimize the relationship of the expression (1) is performed. For this process, the least square method is used. W0 and t0 are obtained by the equation (2) obtained from the least square method.

However, elements other than n in Formula (2) are as follows.

When the condition of the following formula (3) is satisfied, W0 and t0 are as shown in formula (4).

  Here, since the weight w0 and the offset t0 are obtained on the thumbnail image, the amount of data to be processed is reduced and high-speed processing becomes possible.

  Further, even if x [i] and y [i] are distributed as shown in FIG. 4, the weight W0 and the offset t0 are obtained by the least square method, so that they can be obtained accurately and Equation 1 is reduced. be able to.

(Weighted motion vector search unit 5)
The weighted motion vector search unit 5 multiplies the pixel value of the reference image input from the reference image input terminal 9 by the weight w0 obtained by the first weight estimation unit 4 and adds the offset t0 to obtain a new pixel value. Create an attached reference image. Then, a motion vector with the original image from the original image input terminal 8 is searched for this new weighted reference image. An existing technique can be used for this motion vector search.

(Motion compensation unit 6)
The motion compensation unit 6 takes in the motion vector from the weighted motion vector search unit 5 and the reference image from the reference image input terminal 9 and creates a motion compensation image from these motion vectors and the reference image. An existing technique can be used to create the motion compensation image.

(Second weight estimation unit 7)
The second weight estimation unit 7 uses the original image from the original image input terminal 8 instead of the thumbnail original image from the first thumbnail image generation unit 2 and uses the original image from the second thumbnail image generation unit 3 instead of the thumbnail reference image. The motion compensation image from the motion compensation unit 6 is used. The weight w1 and the offset t1 are finally obtained from the original image and the motion compensated image by the same process as the process in the first weight estimation unit 4.

[effect]
As described above, according to the present embodiment, the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 generate thumbnail images of the original image and the reference image, and perform processing based on the thumbnail images. The size of the target image data handled by the one-weight estimation unit 4 can be reduced, and the amount of processing and the processing time can be reduced.

  Further, since the first weight estimation unit 4 uses the least square method, the weight and the offset can be accurately predicted. Furthermore, since the first weight estimation unit 4 and the second weight estimation unit 7 perform weight estimation processing in two stages, the weight and offset can be predicted more accurately.

  By using the weight and offset predicted by the first weight estimation unit 4 and performing a motion vector search by the weighted motion vector search unit 5, it is possible to accurately find a motion vector when the weight is applied. Since the weight and the offset are estimated from the motion compensated image obtained after the motion vector search in the weighted motion vector search unit 5 and the original image from the original image input terminal 8, more accurate estimation can be performed. It becomes like this.

  Furthermore, since an appropriate weight and offset can be obtained using the least square method, the present invention can also be applied to a case where X [i] and Y [i] have a negative correlation as shown in FIG. . In the case of an image change in which a bright part becomes dark and a dark part becomes bright, such a correlation is obtained. In this case, too, estimation can be performed with high accuracy.

[Second Embodiment]
Next, a moving picture coding apparatus and a moving picture coding method according to the second embodiment will be described. In this embodiment, input / output to the weighted motion vector search unit 5, the motion compensation unit 6, and the second weight estimation unit 7 is different from that in the first embodiment. The details of this embodiment will be described below with reference to FIG. Similarly to the moving picture coding apparatus and the moving picture coding method of the first embodiment, the moving picture coding apparatus and the moving picture coding method of the present embodiment are finally input when an original image and a reference image are input. Outputs weights and offsets. The configuration will be described with reference to FIG. 2 in order from the input side.

  As in the first embodiment, the original image input terminal 8 and the reference image input terminal 9 each input an original image to be compressed and a reference image used for compression.

(First thumbnail image generation unit 2 and second thumbnail image generation unit 3)
The original image input from the original image input terminal 8 is sent to the first thumbnail image generation unit 2. In the first thumbnail image generation unit 2, the original image is reduced, and the original image is used as a thumbnail original image. The first weight estimation unit 4, the weighted thumbnail motion vector search unit 5-1, and the second weight estimation unit 7-1. And output respectively.

  The reference image input from the reference image input terminal 9 is sent to the second thumbnail image generation unit 3. The second thumbnail image generation unit 3 reduces the reference image and uses it as a thumbnail reference image, and uses the first weight estimation unit 4, the weighted thumbnail motion vector search unit 5-1, and the thumbnail motion compensation unit 6-1. To each output.

(First weight estimation unit 4)
The first weight estimation unit 4 receives the thumbnail original image from the first thumbnail image generation unit 2. Further, the thumbnail reference image is received from the second thumbnail image generation unit 3. Then, the first weight estimation unit 4 estimates the weight and offset using the received thumbnail original image and thumbnail reference image, as in the first embodiment.

  The estimated weight and offset are sent to the weighted thumbnail motion vector search unit 5-1.

(Weighted thumbnail motion vector search unit 5-1)
The weighted thumbnail motion vector search unit 5-1 searches the thumbnail reference image for the motion vector of the thumbnail original image. Unlike the first embodiment, the weighted thumbnail motion vector search unit 5-1 receives the original thumbnail image from the first thumbnail image generation unit 2 and the thumbnail reference image from the second thumbnail image generation unit 3. Further, the first weight estimation unit 4 receives estimated values of weights and offsets. A motion vector is searched based on these data. The weighted thumbnail motion vector search unit 5-1 outputs the thumbnail motion vector as a result of the search to the thumbnail motion compensation unit 6-1.

(Thumbnail motion compensation unit 6-1)
The thumbnail motion compensation unit 6-1 uses the thumbnail reference image and the thumbnail motion vector to generate a thumbnail motion compensation image. The thumbnail motion compensation unit 6-1 receives the thumbnail reference image from the second thumbnail image generation unit 3. The thumbnail motion vector is received from the weighted thumbnail motion vector search unit 5-1. Then, using the thumbnail reference image and the thumbnail motion vector, a thumbnail motion compensation image is generated and output to the second weight estimation unit 7-1.

(Second weight estimation unit 7-1)
The second weight estimation unit 7-1 receives the thumbnail motion compensation image from the thumbnail motion compensation unit 6-1 and receives the thumbnail original image from the first thumbnail image generation unit 2. The second weight estimation unit 7 obtains weights and offsets from the received thumbnail motion compensation image and thumbnail original image, and outputs the weights and offsets to the weight and offset output terminals.

[Video coding method]
The process of inputting the original image from the original image input terminal 8 to the first thumbnail image generating unit 2 and generating the thumbnail original image by the first thumbnail image generating unit 2 is the same as that in the first embodiment, and therefore will be omitted.

  Since the reference image is input from the reference image input terminal 9 to the second thumbnail image generation unit 3 and the thumbnail reference image is generated by the second thumbnail image generation unit 3 is the same as in the first embodiment, the description is omitted here. . Further, the processing of the first weight estimation unit 4 that receives the thumbnail original image and the thumbnail reference image and outputs the estimated values of the weight and the offset is the same as that in the first embodiment, and is therefore omitted here.

(Weighted thumbnail motion vector search unit 5-1)
The same operation as in the first embodiment is performed except that the original thumbnail image from the first thumbnail image generation unit 2 is used as the input of the reference image from the reference image input terminal 9 instead of the original image from the original image input terminal 8. Instead, it receives a thumbnail reference image from the second thumbnail image generation unit 3. The weighted thumbnail motion vector search unit 5-1 outputs the searched thumbnail motion vector to the thumbnail motion compensation unit 6-1.

(Thumbnail motion compensation unit 6-1)
The thumbnail motion compensator 6-1 basically operates in the same manner as the motion compensator 6 of the first embodiment, but is not a reference image from the reference image input terminal 9 as an input, but a second thumbnail image generator 3. The difference is that it receives thumbnail reference images from.

(Second weight estimation unit 7-1)
The operation is basically the same as that of the second weight estimation unit 7 of the first embodiment, but the original image from the original image input terminal 8 among the inputs is the thumbnail original image from the first thumbnail image generation unit 2. The point is different from the first embodiment.

  Other operations are the same as those in the first embodiment.

[effect]
As described above, according to the second embodiment, the processing of the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 reduces the data amount of the target image handled by the first weight estimation unit 4. Thus, the processing time can be shortened.

  Further, the first weight estimation unit 4 can accurately predict the weight and the offset by using the least square method.

  By performing the motion vector search by the weighted thumbnail motion vector search unit 5-1 using the predicted weight and offset, it is possible to follow a rough motion when a moving part is included in the image.

  In the weighted thumbnail motion vector search unit 5-1, since the thumbnail original image from the first thumbnail image generation unit 2 and the thumbnail reference image from the second thumbnail image generation unit 3 are input, the image size to be processed is small. Thus, the search process is reduced and the processing time is shortened.

  Since the weight and offset are estimated by the second weight estimator 7-1 from the thumbnail motion compensated image obtained by the thumbnail motion compensator 6-1 after the motion vector search and the original image, the moving part is included in the image. Even if it is, it will be possible to estimate with high accuracy.

  Since the input to the second weight estimation unit 7-1 is the thumbnail original image and the thumbnail motion compensation image, the size of the image to be processed by the second weight estimation unit 7-1 is small, the processing is reduced, and the processing time is reduced. Shortening is achieved.

  Furthermore, since an appropriate weight and offset can be obtained using the least square method, the present invention can also be applied to a case where X [i] and Y [i] have a negative correlation as shown in FIG. . In the case of an image change in which a bright part becomes dark and a dark part becomes bright, such a correlation is obtained. In this case as well, the moving picture encoding apparatus of the present embodiment can be applied.

[Third Embodiment]
Next, a moving picture coding apparatus and a moving picture coding method according to the third embodiment will be described with reference to FIG. The configuration of the third embodiment is obtained by adding a third thumbnail image generation unit 11 to the first embodiment. Thereby, the input to the 2nd weight estimation part 7 differs from 1st Embodiment.

  Hereinafter, differences from the first embodiment will be described.

  The third thumbnail image generation unit 11 receives the motion compensated image that is output from the motion compensation unit 6, generates a thumbnail image, and outputs the output to the second weight estimation unit 7.

  In the first embodiment, the original image is input to the second weight estimation unit 7. In the third embodiment, however, the original thumbnail image that is the output from the first thumbnail image generation unit 2 is input instead. . Therefore, the direct input from the original image input terminal 8 to the second weight estimation unit 7 connected in the first embodiment does not exist in the third embodiment.

  In the first embodiment, the output of the motion compensation unit 6 is directly input to the second weight estimation unit 7. In the third embodiment, a thumbnail motion compensation image is received from the output of the third thumbnail image generation unit 11. Therefore, the connection from the output of the motion compensation unit 6 directly to the second weight estimation unit 7 that exists in the first embodiment does not exist in the third embodiment.

[Video coding method]
The operation from the input of the original image and the reference image to the generation of the motion compensated image is the same as in the first embodiment.

(Third thumbnail image generator 11)
The third thumbnail image generator 11 receives the motion compensated image from the motion compensator 6, generates a thumbnail motion compensated image, and outputs the output to the second weight estimator 7.

(Second weight estimation unit 7)
The second weight estimation unit 7 receives the thumbnail original image output from the first thumbnail image generation unit 2 and the thumbnail motion compensation image output from the third thumbnail image generation unit 11, and performs weight estimation. The weight estimation operation itself is the same as in the first embodiment. The estimated weight and offset values are output to the weight and offset output terminals.

[effect]
In the present embodiment, in addition to the effects of the first embodiment, the second weight estimation unit 7 performs processing using the thumbnail original image and the thumbnail motion compensated image. It can be performed at high speed, and the processing time can be further shortened.

[Fourth Embodiment]
Next, a moving picture coding apparatus and a moving picture coding method according to the fourth embodiment will be described with reference to FIG.

  In the present embodiment, an effectiveness determination unit 12 and an effectiveness determination result output terminal 13 are added to the first embodiment. Thereby, an output for sending the statistical parameter from the second weight estimation unit 7 to the validity determination unit 12 is newly added. Further, the output of the second weight estimation unit 7 is connected to the validity determination unit 12 in order to send the weight parameter from the second weight estimation unit 7 to the validity determination unit 12. Other configurations are the same as those of the moving picture encoding apparatus of the first embodiment.

  With the above configuration, the moving image encoding method is as follows. Here, the description will focus on the differences from the first embodiment.

In the present embodiment, the statistical parameter is output from the second weight estimation unit 7 to the validity determination unit 12. The statistical parameters are as follows.

  Further, the second weight estimation unit 7 sends weight parameters (weight W0, offset t0) to the validity determination unit 12.

  The statistical parameter is calculated at the time of estimation by the second weight estimation unit 7, and it is not necessary to perform a new calculation in order to obtain this statistical parameter.

The validity determination unit 12 calculates the determination parameter p as follows.

  Here, n is the number of pixels of the image to be processed.

This formula is as follows.

Therefore, if the statistical parameters input to the validity determination unit are arranged and applied, the following is obtained.

  It can be seen that the amount of calculation can be reduced by using the statistical parameter.

  When p is smaller than a predetermined threshold, the difference between the motion compensated image and the original image is small. That is, it can be determined that the code amount when the difference between the predicted image created using the motion compensated image and the original image is encoded is small, and the weight parameters (weight w0 and offset t0) are determined to be effective.

  In the present embodiment, only the validity of the weight parameter from the second weight estimation unit 7 is determined. Of course, the weight parameter may be received from a plurality of weight estimation units to determine the effectiveness.

  When it is determined that a plurality of weight parameters are valid, a determination portion may be provided so that the determination parameter p is minimized.

[effect]
It is possible to determine whether or not the obtained weight parameter is effective for code amount reduction. Therefore, if it is not effective for reducing the amount of code, it can be said that encoding is performed without applying a weight without using a parameter, and at the same time, it is not necessary to encode the weight parameter itself. Therefore, the code amount can be reduced.

[Fifth Embodiment]
Next, a moving picture coding apparatus and a moving picture coding method according to the fifth embodiment will be described.

  The moving picture encoding apparatus of this embodiment is configured as shown in FIG. The first weight estimation unit 4 in the first embodiment is connected to the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3, but here the motion vector search unit 14 and the first motion compensation unit 15 are connected. It is connected to the.

  The motion vector search unit 14 has the following processing functions.

  The motion vector search unit 14 searches the reference image for a motion vector of the original image. The motion vector search unit 14 receives an original image from the original image input terminal 8 and receives a reference image from the reference image input terminal 9. A motion vector is searched based on these data. The motion vector search unit 14 outputs a motion vector obtained as a result of the search to the first motion compensation unit 15.

  The first motion compensation unit 15 has the following processing functions.

  The first motion compensation unit 15 generates a motion compensated image using the reference image and the motion vector. The first motion compensation unit 15 receives a reference image from the reference image input terminal 9 and receives a motion vector from the motion vector search unit 14. The first motion compensation unit 15 generates a motion compensated image from these reference images and motion vectors, and outputs the motion compensated image to the first weight estimation unit 4.

  The first weight estimation unit 4 has a processing function similar to that of the first weight estimation unit 4 of the first embodiment.

  The weighted motion vector search unit 5-2 has the following processing functions.

  The weighted motion vector search unit 5-2 searches for the motion vector of the original image with respect to the reference image. The weighted motion vector search unit 5-2 receives the original image from the original image input terminal 8 and receives the reference image from the reference image input terminal 9. Furthermore, a motion vector that is not weighted is received from the motion vector search unit 14. The first weight estimation unit 4 receives weight and offset estimation values. A motion vector is searched based on these data. The weighted motion vector search unit 5-2 outputs the weighted motion vector obtained as a result of the search to the second motion compensation unit 6-2.

  The second motion compensation unit 6-2 has the following processing functions.

  The second motion compensation unit 6-2 generates a motion compensated image using the reference image and the weighted motion vector. The second motion compensation unit 6-2 receives a reference image from the reference image input terminal 9, and receives a weighted motion vector from the weighted motion vector search unit 5-2. The second motion compensation unit 6-2 generates a motion compensated image from the reference image and the weighted motion vector, and outputs the motion compensated image to the second weight estimation unit 7.

  The second weight estimation unit 7 has the following processing functions.

  The second weight estimation unit 7 receives the motion compensated image from the second motion compensation unit 6-2 and receives the original image from the original image input terminal 8. Then, a weight and an offset are obtained from the motion compensated image and the original image. The second weight estimator 7 outputs the weight and offset to the weight and offset output terminal 10 and also outputs to the encoding motion vector search unit 16.

  The encoding motion vector search unit 16 has the following processing functions.

  The encoding motion vector search unit 16 searches the reference image for a motion vector of the original image for encoding. The encoding motion vector search unit 16 receives an original image from the original image input terminal 8 and receives a reference image from the reference image input terminal 9. A motion vector that is not weighted is received from the motion vector search unit 14. A weighted motion vector is received from the weighted motion vector search unit 5-2. The first weight estimation unit 4 receives weight and offset estimation values. A motion vector is searched based on these data. The weighted motion vector search unit 5-2 outputs the weighted motion vector obtained as a result of the search to the second motion compensation unit 6-2.

  Next, a video encoding method using the video encoding apparatus having the above configuration will be described.

  The motion vector search unit 14 searches for a motion vector with the original image from the original image input terminal 8 for the reference image input from the reference image input terminal 9. An existing technique can be used for this motion vector search.

  The first motion compensation unit 15 takes in the motion vector from the motion vector search unit 14 and the reference image from the reference image input terminal 9 and creates a motion compensation image from these motion vectors and the reference image. An existing technique can be used to create the motion compensation image.

  The first weight estimation unit 4 obtains a weight and an offset from the compensated image and the original image from the first motion compensation unit 15 by the same process as the process in the first weight estimation unit 4 of the first embodiment. .

  In the weighted motion vector search unit 5-2, the pixel value of the reference image input from the reference image input terminal 9 is multiplied by the weight w0 obtained by the first weight estimation unit 4, and an offset t0 is added to obtain a new pixel value. Create a weighted reference image. Then, a motion vector with the original image from the original image input terminal 8 is searched for this new weighted reference image. An existing technique can be used for this motion vector search.

  The second motion compensation unit 6-2 takes in the motion vector from the weighted motion vector search unit 5-2 and the reference image from the reference image input terminal 9, and uses the motion vector and the reference image as a motion compensation image. make. An existing technique can be used to create the motion compensation image.

  The second weight estimation unit 7 uses the original image from the original image input terminal 8 and the motion compensated image from the second motion compensation unit 6-2. The weight w1 and the offset t1 are finally obtained from the original image and the motion compensated image by the same process as the process in the first weight estimation unit 4.

  Further, the encoding motion vector search unit 16 searches for a motion vector of the original image for encoding. The encoding motion vector search unit 16 receives an original image from the original image input terminal 8, receives a reference image from the reference image input terminal 9, receives an unweighted motion vector from the motion vector search unit 14, and assigns a weight. A weighted motion vector is received from the motion vector search unit 5-2, and weight and offset estimation values are received from the second weight estimation unit 7. Then, a motion vector is searched based on these data. The encoding motion vector search unit 16 outputs a weighted motion vector as a result of the search for encoding.

  In the above moving image coding apparatus and moving image coding method, weighting is performed based on the weight and the estimated offset value estimated by the first weight estimating unit 4 through the motion vector searching unit 14 and the first motion compensating unit 15. Since the motion vector search unit 5-2 searches for a motion vector, and after passing through the motion compensation unit 6-2, the weight and offset are estimated by the second weight estimation unit 7, so that the weight and offset can be accurately predicted. Become.

[Industrial applicability]
In each of the above embodiments, the least square method is used for the first weight estimation unit 4 and the second weight estimation unit 7, but the present invention is not limited to this, and all known estimation methods can be used. That is, the least square method and other methods can be used in appropriate combination. The same applies when there are three or more weight estimation units. Also in this case, the same operations and effects as those of the above embodiments can be obtained. Further, the least square method and another method may be used in combination for the first thumbnail image generation unit 2, the second thumbnail image generation unit 3, and the third thumbnail image generation unit 11.

  In the above embodiments, the vertical and horizontal reduction ratios of the thumbnail images are set to the same value, but the vertical and horizontal reduction ratios may be different.

  Further, in each of the above embodiments, a thumbnail image generation unit that creates a thumbnail is provided as the image data reduction unit, but the image data may be reduced by other methods.

  As in the case of a thumbnail image, a part of the entire image may be cut out other than when the entire image is reduced. For example, the screen may be divided into upper and lower thumbnail images only for the upper half or the lower half, and weight estimation may be performed. Alternatively, the image may be divided into four, thumbnail images may be generated for one or more of them, and weight estimation may be performed. If it is difficult to estimate the weight without processing the entire screen, such as the brightness of the screen, a thumbnail image is generated for the entire screen. If the weight can be estimated even if only processing is performed, a part may be cut out and processed. Further, in this case, the weight may be estimated by cutting out only a specific part of the screen, which is the same part of the original image and the reference image, without being limited to a plurality of divisions.

  In addition to the case where the image data reduction unit is a reduced image obtained by summing a plurality of pixels as in the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3, the image data reduction unit is constant among the pixels of the entire image. The amount of data may be reduced by thinning out at intervals. In this case, although the accuracy is somewhat reduced, the data amount of the image can be reduced while leaving the characteristics of the original image.

  Furthermore, a reduction rate adjustment unit that adjusts the reduction rate of the image data in the image data reduction unit may be provided. For example, a reduction rate adjustment unit that can arbitrarily adjust the reduction rate is connected to the first thumbnail image generation unit 2 and the second thumbnail image generation unit 3 to arbitrarily adjust the data reduction rate. Good. In this case, usually, if it is desired to increase the processing speed even if accuracy is lowered, the thumbnail image is reduced by increasing the reduction ratio. Conversely, if you want to increase the accuracy even if you reduce the processing speed, reduce the reduction ratio and do not make the thumbnail image very small. The reduction ratio adjustment unit appropriately adjusts according to various conditions such as usage mode and usage. Thereby, it is possible to easily select an optimum processing condition according to each device used under various conditions.

  In the second embodiment, the first weight estimation unit 4 always outputs the weight parameter (w0, t0) as (w0, t0) = (1, 0), that is, always outputs no weight. It may be configured. This configuration is the same as the configuration in which the first weight estimation unit 4 is not provided and the weighted thumbnail motion vector search unit 5-1 searches without weight, and this configuration is naturally included in the present embodiment.

  In the third embodiment, a configuration in which the first weight estimation unit 4 always outputs the weight parameter (w0, t0) as (w0, t0) = (1, 0), that is, a configuration in which there is always no weight. But you can. This configuration is the same as the configuration in which there is no first weight estimation unit 4, no second thumbnail image generation unit 3 that generates an input to the first weight estimation unit 4, and the weighted motion vector search unit searches without weight. Of course, this configuration is also included in this embodiment.

  Each of the embodiments outputs a weight parameter, but the output weight parameter may be further used as an input to the weighted motion vector search unit of each embodiment to further perform weight estimation. The weighted motion vector search unit that receives the output weight parameter need not be in the same embodiment. The weight parameter that is the output of the above may be input to the weighted motion vector search unit of the second embodiment. The operation of inputting the output weight parameter to the weighted motion vector search unit may be repeated any number of times.

  In each of the embodiments, the weight estimation process in each weight estimation unit is performed by the least square method. However, when a plurality of weight estimation units are provided, the least square method is used for one of the weight estimation units, The weight estimation unit may appropriately use the least square method or other methods. Furthermore, each weight estimation unit may include the validity determination unit, and a weight parameter selection unit that selects an output from the plurality of weight estimation units using the determination result of the validity determination unit. Accordingly, a more accurate output can be selected from a plurality of weight estimation units based on the determination result of the validity determination unit, and the weight parameter becomes more accurate.

  In each of the embodiments, the first thumbnail image generation unit 2, the second thumbnail image generation unit 3, the first weight estimation unit 4, the weighted motion vector search unit 5, the motion compensation unit 6, the second weight estimation unit 7, and the like are included. The moving image coding apparatus is configured by combining the two components of the image data reduction unit and the weight estimation unit, the image data reduction unit, the weighted motion vector search unit, the motion compensation unit, and the weight estimation unit. You may comprise only one component. The above configuration may be used when a simple configuration is desired even if the accuracy is somewhat reduced.

It is a schematic block diagram which shows the moving image encoder which concerns on 1st Embodiment of this invention. It is a schematic block diagram which shows the moving image encoder which concerns on 2nd Embodiment of this invention. It is a schematic block diagram which shows the moving image encoder which concerns on 3rd Embodiment of this invention. It is a graph explaining the conventional weight estimation method. It is a graph explaining the subject of the conventional weight estimation method. It is a graph explaining the case where a correlation is negative. It is a functional block diagram which shows the moving image encoding device for implement | achieving the moving image encoding method which concerns on 4th Embodiment. It is a functional block diagram which shows the moving image encoding device for implement | achieving the moving image encoding method which concerns on 5th Embodiment.

Explanation of symbols

  1: moving image encoding device, 2: first thumbnail image generation unit, 3: second thumbnail image generation unit, 4: first weight estimation unit, 5: weighted motion vector search unit, 6: motion compensation unit, 7 : Second weight estimation unit, 8: original image input terminal, 9: reference image input terminal, 11: third thumbnail image generation unit, 12: validity determination unit, 13: validity determination result output terminal, 14: motion vector Search unit, 15: first motion compensation unit, 16: motion vector search unit for encoding.

Claims (29)

In a moving image encoding apparatus that encodes a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction unit that reduces the amount of data of the original image and the reference image while leaving the features of the original image and the reference image;
A video encoding apparatus comprising: a weight estimation unit that estimates a weight parameter based on image data of an original image and a reference image whose data amount has been reduced by the image data reduction unit. The moving image encoding device according to claim 1,
The moving image coding apparatus, wherein the weight estimation unit processes the image data by a least square method. The moving image encoding device according to claim 1 or 2,
The moving image encoding apparatus, wherein the image data reduction unit includes a thumbnail image generation unit for generating a thumbnail. In a video encoding device that encodes a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction unit that reduces the amount of data of the original image and the reference image while leaving the features of the original image and the reference image;
A weighted motion vector search unit that searches for a motion vector based on the original image and the reference image reduced by the image data reduction unit;
A motion compensation unit that generates a motion compensated image using the motion vector searched by the weighted motion vector search unit and the reference image;
A video encoding apparatus comprising: a motion estimation image generated by the motion compensation unit; and a weight estimation unit that estimates a weight parameter from the original image. In a video encoding device that encodes a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction unit that reduces the amount of data of the original image and the reference image while leaving the features of the original image and the reference image;
A first weight estimation unit that estimates weight parameters based on the image data of the original image and the reference image whose data amount has been reduced by the image data reduction unit;
A weighted motion vector search unit that searches for a motion vector based on the weight parameter estimated by the first weight estimation unit and the original image and the reference image before or after being reduced by the image data reduction unit;
A motion compensation unit that generates a motion compensated image using the motion vector searched by the weighted motion vector search unit and the reference image;
A moving picture coding apparatus comprising: a motion compensation image generated by the motion compensation unit; and a second weight estimation unit that estimates a weight parameter from the original image. In the moving image encoding device according to claim 4,
The moving image coding apparatus, wherein the motion compensation unit performs processing based on original image data and reference image data after being reduced by the image data reduction unit. The moving image encoding device according to claim 5,
The moving image coding apparatus, wherein the motion compensation unit and the second weight estimation unit perform processing based on original image data and reference image data after being reduced by the image data reduction unit. The moving image encoding device according to claim 5,
A motion-compensated image data reducing unit that reduces the amount of data while retaining the features of the motion-compensated image from the motion-compensating unit; and the second weight estimating unit includes image data from the motion-compensated image data reducing unit; A video encoding apparatus, wherein the video data is processed based on the original image data reduced by the image data reduction unit. The moving image encoding device according to any one of claims 4 to 8,
The moving image encoding apparatus, wherein the image data reduction unit includes a thumbnail image generation unit for generating a thumbnail. The moving image encoding apparatus according to claim 8, wherein
The moving image encoding apparatus, wherein the motion compensation image data reduction unit includes a thumbnail image generation unit for generating a thumbnail. The moving image encoding device according to any one of claims 5 to 10,
One or both of said 1st weight estimation part or said 2nd weight estimation part process using the least squares method, The moving image coding apparatus characterized by the above-mentioned. The moving picture encoding device according to any one of claims 1 to 11,
A moving picture encoding apparatus, further comprising: a reduction rate adjustment unit that adjusts a reduction rate of image data in the image data reduction unit. The moving image encoding device according to any one of claims 8 to 11,
A moving picture coding apparatus, further comprising: a reduction rate adjustment unit that adjusts a reduction rate of image data in the motion compensated image data reduction unit. In a video encoding device that encodes a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
A weight estimation unit for obtaining a weight parameter from the reference image and the original image by a least square method;
A moving picture coding apparatus comprising an effectiveness determination unit that determines the validity of the weight and offset output from the weight estimation unit using the statistics generated in the weight calculation process of the weight estimation unit . The moving picture encoding apparatus according to claim 14,
In addition to the weight estimation unit by the least square method, one or more weight estimation units for performing weight estimation processing by the least square method or other methods are provided,
A moving picture coding apparatus comprising: a weight parameter selection unit that selects a plurality of outputs from the weight estimation unit using a determination result of the validity determination unit. In a video encoding method for encoding a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction process for reducing the data amount of the original image and the reference image while leaving the features of the original image and the reference image;
A moving image encoding method comprising: a weight estimation step of estimating a weight parameter based on image data of an original image and a reference image whose data amount has been reduced in the image data reduction step. The video encoding method according to claim 16, wherein
The moving image encoding method, wherein the weight estimation step processes the image data by a least square method. In a video encoding method for encoding a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction process for reducing the data amount of the original image and the reference image while leaving the features of the original image and the reference image;
A weighted motion vector search step for searching for a motion vector based on the original image and the reference image reduced in the image data reduction step;
A motion compensation step of generating a motion compensation image using the motion vector searched in the weighted motion vector search step and the reference image;
A moving image encoding method comprising: a weight estimation step of estimating a weight parameter from the motion compensation image generated in the motion compensation step and the original image. In a video encoding method for encoding a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
An image data reduction process for reducing the data amount of the original image and the reference image while leaving the features of the original image and the reference image;
A first weight estimation step of estimating a weight parameter based on the image data of the original image and the reference image whose data amount has been reduced in the image data reduction step;
A weighted motion vector search step for searching for a motion vector based on the weight parameter estimated in the first weight estimation step and the original image and the reference image before or after being reduced by the image data reduction unit;
A motion compensation step of generating a motion compensation image using the motion vector searched in the weighted motion vector search step and the reference image;
A moving image encoding method comprising: a second weight estimation step of estimating a weight parameter from the motion compensation image generated in the motion compensation step and the original image. The video encoding method according to claim 18, wherein
The moving image encoding method, wherein the motion compensation step performs processing based on the original image data and the reference image data after being reduced in the image data reduction step. The moving image encoding method according to claim 19,
The moving picture encoding method, wherein the motion compensation step and the second weight estimation step are processed based on the original image data and the reference image data after being reduced in the image data reduction step. The moving image encoding method according to claim 19,
A motion compensation image data reduction step for reducing the amount of data while retaining the features of the motion compensation image from the motion compensation step, and the second weight estimation step includes image data from the motion compensation image data reduction step; A moving image encoding method, wherein the processing is performed based on original image data that has been reduced in the image data reduction step. The moving image encoding method according to any one of claims 18 to 22,
A moving image encoding method, wherein the image data reduction step comprises a thumbnail image generation step of creating a thumbnail. The moving image encoding method according to claim 22,
A moving image encoding method, wherein the motion compensated image data reduction step includes a thumbnail image generation step of creating a thumbnail. The moving image encoding method according to any one of claims 19 to 24,
One or both of said 1st weight estimation process or said 2nd weight estimation process process using the least squares method, The moving image coding method characterized by the above-mentioned. The moving image encoding method according to any one of claims 16 to 25,
A moving picture encoding method comprising a step of adjusting a reduction rate of image data in the image data reduction step according to required processing speed and accuracy. The moving image encoding method according to any one of claims 16 to 25,
A moving picture encoding method comprising a step of adjusting a reduction rate of image data in the motion compensated image data reduction step according to required processing speed and accuracy. In a video encoding method for encoding a difference between a predicted image generated using a weighted reference image obtained by applying a weight to a reference image and adding an offset, and the original image,
A weight estimation step of obtaining a weight parameter from the reference image and the original image by a least square method;
A video encoding method comprising a validity determination step for determining the validity of the weight and offset output in the weight estimation step using the statistics generated in the weight calculation processing step of the weight estimation step . The video encoding method according to claim 28, wherein
In addition to the weight estimation step by the least square method, one or more weight estimation steps for performing weight estimation processing by the least square method or other methods are provided.
A moving picture encoding method comprising a weight parameter selection step of selecting a plurality of outputs from the weight estimation step using the determination result in the validity determination step.

Images