JP2015195470A - encoding device, encoding method and encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make reduction of a processing time and suppression of image quality deterioration compatible for generating encoded data which can be reproduced with a plurality of kinds of screen sizes or frame rates, from one moving image.SOLUTION: An encoding device includes: a search section which searches for a motion vector and/or a prediction direction for each first block included in each of a plurality of first images within a first moving image; an encoding section for encoding a second moving image generated from the first moving image; a conversion section for calculating a motion vector and/or a prediction direction estimated for each of second blocks included in each of a plurality of second images in the second moving image on the basis of the motion vector and/or the prediction direction obtained by the search, and calculating second cost indicating a prediction error estimated for each second block from first cost indicating a prediction error between a prediction image that is indicated by the motion vector and/or the prediction direction obtained by the search, and the first block; and a control section which makes the encoding section encode the second block of which the second cost being equal to or less than a predetermined first threshold is calculated, using the motion vector and/or the prediction direction obtained by the conversion section.

Description

  The present invention relates to an encoding device, an encoding method, and an encoding program.

  In the field of services for distributing moving image content, in order to distribute moving image data to various types of terminal devices used by users to view moving images, a single moving image content can be divided into a plurality of types of screen sizes. There is a technique for encoding at a frame rate. In this type of technology, for example, first encoded data encoded with the screen size of a television receiver is converted into second encoded data that can be viewed on a terminal device having another screen size such as a smartphone. Transcoding to convert may be performed.

  In addition, in order to specify a predictive coding mode for encoding a moving image with high accuracy, an intra-screen prediction mode or an inter-screen prediction mode is selected based on a cost obtained by a prediction process with a small amount of processing, and the selected mode is selected. A technique for encoding a moving image has been proposed (see, for example, Patent Document 1).

JP 2007-243337 A

  By the way, in the conventional technique for specifying the prediction encoding mode used for encoding a moving image, the moving image that is the target of the preceding stage encoding process and the moving image that is the target of the subsequent stage encoding process are the same. It is assumed that. Therefore, when the video size encoded in the subsequent encoding process and the video image used in the preceding prediction process have different screen sizes and frame rates, the mode used for encoding using the above-described conventional technology It is difficult to specify.

  On the other hand, when the original moving image data is generated by encoding each type of terminal device in order to provide moving images to a plurality of types of terminal devices having different screen sizes and frame rates, one type of encoded data It takes time to multiply the time for generating the number of types of terminal devices.

  However, for example, when encoded data for a television receiver is converted into encoded data for a smartphone, the motion vector obtained by the conversion performs a motion search for an image with a reduced screen size. There is a case where it deviates from the obtained motion vector. In this case, the image quality of the image restored from the encoded data obtained by transcoding can be obtained by encoding using the motion vector obtained by the motion search for the image after reducing the screen size. The quality of the image restored from the encoded data is lowered.

  The encoding apparatus, encoding method, and encoding program disclosed herein reduce processing time and image quality degradation for generating encoded data that can be reproduced from a single moving image at a plurality of types of screen sizes and frame rates. It aims at providing the technology which makes suppression compatible.

  According to one aspect, the encoding device performs inter-screen prediction and intra-screen prediction for each of the plurality of first blocks obtained by dividing each of the plurality of first images included in the first moving image. A search unit that searches for at least one of a motion vector and a prediction direction by performing at least one, and a second generated at least one of a frame rate and a screen size that is generated from the first moving image is different from that of the first moving image. An encoding unit that encodes each of a plurality of second images included in the moving image for each second block obtained by dividing the second image, and at least a motion vector and a prediction direction obtained by the search unit From each of the first information indicating one and the first cost indicating the prediction error that is the difference between the predicted image indicated by the first information and the first block, the second block included in each second image. Second information indicating at least one of a motion vector and a prediction direction that are expected to be obtained when at least one of inter-screen prediction and intra-screen prediction is performed for each of the two, a predicted image indicated by the second information, and a second When encoding a conversion unit that obtains each of a second cost indicating a prediction error expected between the block and a second block for which the conversion unit obtains a second cost that is equal to or less than a predetermined first threshold. And a control unit that causes the encoding unit to encode using the second information obtained by the conversion unit.

  According to another aspect, the encoding method performs inter-screen prediction and intra-screen prediction for each of a plurality of first blocks obtained by dividing each of a plurality of first images included in the first moving image. By performing at least one of the predictions, at least one of the motion vector and the prediction direction is searched, and at least one of the frame rate and the screen size generated from the first moving image is different from the first moving image. When encoding each of the plurality of second images included in the moving image for each second block obtained by dividing the second image, the second image indicating at least one of the motion vector obtained by the search and the prediction direction. Each of the second block included in each second image from one information and a first cost indicating a prediction error that is a difference between the predicted image indicated by the first information and the first block. Second information indicating at least one of a motion vector and a prediction direction expected to be obtained when at least one of inter-screen prediction and intra-screen prediction is performed; a predicted image indicated by the second information; and a second block Each of the second cost indicating a prediction error expected during the period is obtained, and the second information obtained for the second block is used for the second block for which the second cost equal to or lower than the predetermined first threshold is obtained. To encode.

  Further, according to another aspect, the encoding program performs inter-screen prediction and intra-screen prediction for each of the plurality of first blocks obtained by dividing each of the plurality of first images included in the first moving image. By performing at least one of the predictions, at least one of the motion vector and the prediction direction is searched, and at least one of the frame rate and the screen size generated from the first moving image is different from the first moving image. When encoding each of the plurality of second images included in the moving image for each second block obtained by dividing the second image, the second image indicating at least one of the motion vector obtained by the search and the prediction direction. 1 information and the first cost indicating the prediction error, which is the difference between the predicted image indicated by the first information and the first block, of the second block included in each second image. Second information indicating at least one of a motion vector and a prediction direction expected to be obtained when at least one of inter-screen prediction and intra-screen prediction is performed for each, a predicted image indicated by the second information, and a second The second information obtained for the second block is obtained with respect to each of the second costs indicating the prediction error expected between the blocks and the second block for which the second cost equal to or lower than the predetermined first threshold is obtained. The computer executes the process of encoding using.

  The encoding apparatus, encoding method, and encoding program disclosed herein reduce processing time and image quality degradation for generating encoded data that can be reproduced from a single moving image at a plurality of types of screen sizes and frame rates. It is possible to achieve both suppression.

It is a figure which shows one Embodiment of an encoding apparatus. It is a figure which shows the example of the correspondence of the macroblock of a reduction image, and the macroblock before reduction. It is a figure which shows operation | movement of the encoding apparatus shown in FIG. It is a figure which shows the example of conversion of the motion vector by the conversion part shown in FIG. It is a figure which shows the example of a conversion of the prediction direction by the conversion part shown in FIG. It is a figure which shows another embodiment of an encoding apparatus. It is a figure which shows the example of the difference of the 2nd cost compared with a 2nd threshold value in the suppression part shown in FIG. It is a figure which shows an example of the hardware constitutions of the encoding apparatus shown in FIG. It is a figure which shows operation | movement of the encoding apparatus shown in FIG.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 shows an embodiment of an encoding device. The encoding apparatus 10 illustrated in FIG. 1 includes a search unit 11, an encoding unit 12, a conversion unit 13, and a control unit 14.

  The search unit 11 and the encoding unit 12 illustrated in FIG. 1 are connected to the moving image database DB and receive the moving image Vin stored in the moving image database DB. The moving image Vin is, for example, a moving image obtained by continuously photographing a subject at a predetermined time interval, and includes a plurality of images. The encoding unit 12 is connected to the distribution device DT, and the encoded data C1 to Cn (n is an integer of 2 or more) generated by the encoding unit 12 is sent to the network NW via the distribution device DT. And distributed to each of the n types of terminal apparatuses UE. The television device UE1, the personal computer UE2, the tablet terminal UE3, and the smart phone UE4 illustrated in FIG. 1 receive n types of terminal devices UE that receive the respective distributions of the encoded data C1 to Cn generated by the encoding unit 12. It is an example.

  Note that the n types of terminal devices UE are not limited to the television device UE1, the personal computer UE2, the tablet terminal UE3, and the smart phone UE4, and may be any terminal device that has a function of reproducing moving images. Or a portable game machine. Also, each of the encoded data C1 to Cn generated by the encoding unit 12 included in the encoding device 10 may be distributed in parallel to a plurality of terminal devices UE of the same type by the distribution device DT. In addition, the moving image database DB, the encoding device 10, and the distribution device DT may be individual devices as shown in FIG. 1, or may be integrated as one device.

  In the encoding device 10 illustrated in FIG. 1, the search unit 11 performs, for example, at least one of inter-screen prediction and intra-screen prediction for each macroblock obtained by dividing each of a plurality of images included in the moving image Vin. Thus, at least one of the motion vector and the prediction direction is searched. Here, each of the plurality of images included in the moving image Vin is an example of a first image to be searched by the search unit 11. The macroblock included in each image in the moving image Vin is an example of a plurality of first blocks obtained by dividing the first image.

  The search unit 11 is, for example, H.264. Based on the H.264 standard, the motion vector and the prediction direction of each sub block may be searched by performing intra prediction and inter prediction for each sub block obtained by dividing each macro block.

  In the following, for simplicity of explanation, the search unit 11 uses, for each macroblock of each image included in the moving image Vin, a screen in which an image taken before the image including the macroblock in the moving image Vin is used as a reference image. A case where a motion vector is searched by inter prediction will be described. For example, when an image number indicating the shooting order is given to each image included in the moving image Vin, the search unit 11 uses the motion vector of each macroblock included in the image of the image number k as the image number k−1. Search is performed by performing inter-screen prediction using an image as a reference image.

  In this case, the search unit 11 sequentially pays attention to the macroblocks of each image included in the moving image Vin, and pays attention to the reference image that is the previous image in the shooting order of the image including the target macroblock. Search for a place similar to a macroblock. Then, with reference to the position corresponding to the macroblock of interest in the reference image, a motion vector indicating the position of the location specified by the search is obtained, and the first information including the obtained motion vector and information indicating the reference image is converted into the conversion unit 13. To pass. In addition, the search unit 11 determines, for example, the sum of absolute values of differences between the image of the location specified by the search in the reference image and the macroblock of interest by the difference between the predicted image indicated by the motion vector and the macroblock of interest. The first cost indicating a certain prediction error is passed to the conversion unit 13.

  The encoding unit 12 illustrated in FIG. 1 generates, for example, a moving image Vin based on an encoding condition set for each type of terminal device UE as a distribution destination. The moving image Vin includes a frame rate and a screen size. A second moving image in which at least one of them is different is encoded. For example, the encoding unit 12 thins out an image included in the moving image Vin in accordance with the frame rate indicated by the encoding condition, and also performs image processing according to the screen size indicated by the encoding condition. A second moving image is generated by reducing each of them. Then, in accordance with an instruction from the control unit 14, the encoding unit 12 obtains by performing inter-screen prediction for each macroblock in each of the reduced images that are the plurality of second images included in the second moving image. Encoding using either the information obtained or the information obtained by the conversion unit 13 is performed. Here, the image reduced according to the screen size indicated by the encoding condition for each type of the terminal device UE as the distribution destination is a second moving image that is different from the moving image Vin in at least one of the frame rate and the screen size. It is an example of the 2nd image contained. Each macroblock included in the reduced image is an example of a plurality of second blocks obtained by dividing the second image.

  In addition, the ratio of the size of the first image and the second image set in the encoding condition for each type of the terminal device UE is the size of the image that can be displayed on the display unit mounted on each of the terminal devices UE. Based on this, the vertical direction and the horizontal direction of the image may be set independently.

  In the following description, an image reduced according to the screen size indicated by the encoding condition set for each type of terminal apparatus UE as a distribution destination may be referred to as a reduced image. In the following description, for simplification of description, the encoding unit 12 does not perform image thinning from the moving image Vin, and all images included in the moving image Vin are based on the screen size specified by the encoding condition. Will be described. Further, the ratio of the size of the second image to the first image designated by the encoding condition is referred to as a reduction ratio.

  The conversion unit 13 illustrated in FIG. 1 uses a motion vector included in the first information received from the search unit 11 to display a screen between screens using a reduced image that is the previous reduced image in the shooting order for each macroblock of the reduced image. Second information indicating a motion vector expected to be obtained by prediction is obtained. For example, the conversion unit 13 specifies, for each macroblock of the reduced image, the pre-reduction macroblock associated with the macroblock based on the reduction ratio specified by the encoding condition. Then, for example, the conversion unit 13 reduces the motion vector obtained for each of the specified macroblocks using the specified reduction ratio, and selects any one of the reduced motion vectors, thereby reducing the reduced image. The motion vector of the macroblock is obtained. The conversion unit 13 obtains a motion vector predicted for each macroblock of the reduced image by any of vectors obtained by reducing the motion vector of the macroblock before reduction specified corresponding to the macroblock of the reduced image. It is not restricted to the method of selecting. For example, the conversion unit 13 weights and averages a vector obtained by weighting a motion vector of a macroblock before reduction specified corresponding to the macroblock of the reduced image and reducing the macroblock of the reduced image. You may obtain | require as a motion vector of a block. An example suitable as a weight used when a motion vector predicted for a macroblock of a reduced image is obtained by weighted averaging will be described later with reference to FIG.

  For example, the conversion unit 13 determines, for each macroblock of the reduced image, between the predicted image indicated by the motion vector obtained as the second information and the macroblock of the reduced image from the first cost received from the search unit 11. A second cost indicating a prediction error expected is obtained. For example, the conversion unit 13 increases in accordance with the difference in direction between the motion vector of the macroblock before reduction associated with the macroblock of the reduced image and the motion vector obtained for the macroblock of the reduced image. Find a coefficient with a value. Then, for example, the conversion unit 13 multiplies the first cost received from the search unit 11 corresponding to each of the macroblocks before reduction by the obtained coefficient and reduction rate, and first multiplied by the coefficient and reduction rate. The second cost is obtained by adding the costs to each other.

  Here, the macroblocks before reduction specified corresponding to the macroblocks of the reduced image are adjacent to each other in the image before reduction as described later with reference to FIG. 2, for example. The motion vectors of macroblocks are often similar to each other. When the motion vectors of the identified macroblocks are similar to each other, the vectors obtained by reducing these motion vectors at the image reduction rate are actually obtained when inter-screen prediction is performed for the macroblocks of the reduced image Is likely to be similar to the motion vector obtained. That is, when the motion vectors of the specified macroblocks are similar to each other, the motion vector obtained from the motion vector of each macroblock specified by the conversion unit 13 is actually between the screens for the macroblock of the reduced image. This is almost the same as when the prediction is made.

  In addition, the second cost obtained for each macroblock of the reduced image by the conversion unit 13 described above is when the motion vectors of the macroblocks before reduction associated with the macroblock of the reduced image are similar to each other. The value is smaller than in other cases. That is, as the second cost obtained by the conversion unit 13 is smaller, the motion vector obtained as the second information by the conversion unit 13 is more likely as the motion vector of the macroblock of the reduced image.

  Therefore, the control unit 14 illustrated in FIG. 1, when the second cost received from the conversion unit 13 for each macroblock of the reduced image is equal to or less than a predetermined first threshold, the motion vector indicated by the second information, It is determined that the motion vector actually obtained by the inter-screen prediction for the reduced image is similar. Here, the 1st threshold value compared with the 2nd cost in control part 14 is set as the average value of the prediction error obtained by inter-screen prediction, for example. Note that the first threshold value achieves specifications for image quality and processing speed that are set in advance for each terminal device UE for each terminal device UE that is a distribution destination of encoded data generated by the encoding unit 12. It may be set to a possible value.

  Then, the control unit 14 uses the motion vector indicated by the second information received from the conversion unit 13 to the encoding unit 12 when the second cost is equal to or lower than the predetermined first threshold. Is instructed to execute. In this case, the encoding unit 12 omits the process of obtaining the motion vector for the macroblock of the reduced image, and encodes the macroblock of the reduced image using the motion vector obtained by the conversion by the conversion unit 13. As described in the conversion unit 13, the smaller the second cost is, the more likely the motion vector obtained as the second information by the conversion unit 13 is as the motion vector of the macroblock of the reduced image. Therefore, when the second cost is equal to or lower than the first threshold, the control unit 14 causes the encoding unit 12 to perform encoding using the motion vector indicated by the second information, thereby reducing the macroblock of the reduced image. Probable motion vectors can be used for encoding.

  On the other hand, when the second cost is greater than the first threshold, the control unit 14 performs inter-screen prediction on the macroblock of the reduced image for the motion vector included in the second information obtained by the conversion unit 13. It is determined that the obtained motion vector may be different. In this case, the control unit 14 instructs the encoding unit 12 to execute inter-screen prediction on the macroblock of the reduced image, and executes encoding using the motion vector obtained by the inter-screen prediction. .

  Here, the correspondence between the macroblock of the reduced image and the macroblock in the image before reduction will be described with reference to FIG.

  FIG. 2 shows an example of a correspondence relationship between a macroblock of a reduced image and a macroblock before reduction. FIG. 2A shows a case where four macroblocks MB11, MB12, MB21, and MB22 in the image before reduction are associated with one macroblock mb11 in the reduction image. FIG. 2B shows a case where a region R including a part of each of the four macroblocks MB11, MB12, MB21, and MB22 in the image before reduction is associated with one macroblock mb11 in the reduction image. Show.

  In the example of FIG. 2A, the vector Vc obtained by reducing the motion vector Vp11 of the macroblock MB11 in the upper left among the macroblocks MB11 to MB22 by the conversion unit 13 illustrated in FIG. 1 is used as the motion vector of the macroblock mb11. Indicates when it is selected. In FIG. 2A, solid arrows Vp12, Vp21, and Vp22 indicate motion vectors obtained for the macroblocks MB12, MB21, and MB22, respectively, by the search unit 11 shown in FIG. Also, in the macroblocks MB12, MB21, and MB22 shown in FIG. 2A, vectors corresponding to the motion vector Vc of the macroblock mb11 are indicated by dotted arrows.

  In this case, for example, for each of the motion vectors Vp11 to Vp22, the conversion unit 13 obtains a coefficient indicating the magnitude of the difference in direction from the motion vector Vc, and uses it for the process of obtaining the second cost of the macroblock mb11. It is done. In the example of FIG. 2A, each of the macroblocks MB11 to MB22 is associated with the macroblock mb11. In this case, the conversion unit 13 calculates the sum of the products of the coefficient indicating the difference in direction and the image reduction ratio obtained for each of the motion vectors Vp11 to Vp22 and the first cost of each of the macroblocks MB11 to MB22. Obtained as the second cost of the macroblock mb11.

  On the other hand, the example of FIG. 2B shows a case where the proportions of the portions associated with the macroblock mb11 are different from each other in the macroblocks MB11 to MB22 before the reduction. In this case, the conversion unit 13 performs the weighted average using the weight indicating the ratio between the area of the portion included in the region R and the area of one macroblock in each of the macroblocks MB11 to MB22, and the macroblock mb11 The motion vector Vc may be obtained. In the macroblocks MB11, MB12, MB21, and MB22 shown in FIG. 2B, the dotted arrow Vc indicates the motion vector of the macroblock mb11 obtained by the weighting average of the motion vectors Vp11 to Vp22 by the conversion unit 13. Indicates. In this case, it is preferable that the conversion unit 13 calculates the second cost using the weight used in the calculation of the motion vector Vc.

  In the encoding device 10 illustrated in FIG. 1, the control unit 14 performs the movement indicated by the second information when the second cost obtained for each macroblock of the reduced image by the conversion unit 13 is equal to or less than the first threshold. The macroblock of the reduced image is encoded using the vector. As a result, when a probable motion vector is obtained as the motion vector of the macroblock of the reduced image by the conversion unit 13, the inter-screen prediction process in the encoding unit 12 is omitted, and the encoding process is speeded up. Can do. As described above, when the second cost obtained for an arbitrary macroblock of the reduced image is equal to or smaller than the first threshold, the motion vector indicated by the second information obtained by the conversion unit 13 is the macroblock. Is almost equivalent to the motion vector obtained by the inter-screen prediction. Therefore, when the second cost equal to or lower than the first threshold is obtained, the inter-frame prediction is performed on the macroblock of the reduced image by encoding the macroblock of the reduced image using the motion vector indicated by the second information. Encoded data that can be reproduced with the same image quality as that obtained is obtained. That is, the encoding apparatus 10 shown in FIG. 1 reduces the processing time and the image quality for encoding processing for generating encoded data that can be reproduced at a plurality of types of screen sizes and frame rates from one moving image Vin. It is possible to achieve both suppression of deterioration.

  FIG. 3 shows the operation of the encoding apparatus 10 shown in FIG. The processing of step S301 to step S306 shown in FIG. 3 shows the operation of the encoding device 10 shown in FIG. 1, and shows an example of the encoding method and the encoding program. For example, the processing illustrated in FIG. 3 is realized by a processor installed in the encoding device 10 executing an encoding program. Note that the processing illustrated in FIG. 3 may be executed by hardware installed in the encoding device 10.

  In step S301, the search unit 11 obtains, for example, a motion vector and a prediction error as the first information and the first cost for each macroblock in the image before reduction based on the image before reduction included in the moving image Vin. .

  In step S302, the conversion unit 13 calculates the second information indicating the predicted motion vector and the predicted prediction error for each macroblock of the reduced image based on the motion vector and the prediction error obtained in the process of step S301. The second cost shown is determined. In the process of step S302, the conversion unit 13 uses the motion vector and the prediction error obtained for the macroblock before reduction associated with each macroblock of the reduced image, as described in FIG. Note that the correspondence between each macroblock of the reduced image and the macroblock before the reduction is specified based on a reduction ratio when a reduced image is generated from each image included in the moving image Vin.

  In each of the processes in steps S303 to S306 described below, each macroblock included in the reduced image is set as a macroblock to be noticed in order, and each macroblock to be noticed is converted by the conversion unit 13, the control unit 14, and the encoding unit 12. To be executed.

  In step S303, the control unit 14 determines whether or not the second cost obtained in the process of step S302 is equal to or less than the first threshold for the macro block of interest.

  When the second cost having a value equal to or smaller than the first threshold value is obtained in the process of step S302 (Yes determination in step S303), the control unit 14 pays attention to the motion vector obtained in the process of step S302. It is determined that the motion vector of the macro block is likely. In this case, the control unit 14 proceeds to the process of step S304.

  In step S304, the control unit 14 passes the motion vector obtained in the process of step S302 as the motion vector of the target macroblock to the encoding unit 12, and performs the encoding process using the passed motion vector. 12 is executed. In this case, the encoding unit 12 omits the process of searching for a motion vector for a macroblock for which a second cost having a value equal to or smaller than the first threshold is obtained.

  On the other hand, when the second cost obtained in step S302 is larger than the first threshold value (No determination in step S303 (NO)), the control unit 14 determines that the motion vector obtained in step S302 is the macro block of interest. It is determined that the motion vector is not accurate. In this case, the control unit 14 proceeds to the process of step S305.

  In step S305, the control unit 14 causes the encoding unit 12 to execute, for example, a motion vector search by inter-screen prediction processing for the macro block of interest.

  In step S306, the encoding unit 12 encodes the macro block of interest using the motion vector obtained by the inter-screen prediction process or the like executed in step S305.

  As described above, the encoding apparatus 10 shown in FIG. 1 determines the predicted motion vector when the motion vector of the macroblock of the reduced image predicted by the processing of the conversion unit 13 is likely. Used to encode a macroblock of a reduced image. Here, a moving image in which a person or a landscape is photographed includes many portions where the gradation change is gentle. Then, with respect to the macroblock of the reduced image associated with the macroblock included in the portion where the gradation change is gradual, a probable motion vector can be predicted by the processing of the conversion unit 13 described in FIG. Therefore, the encoding apparatus 10 illustrated in FIG. 1 omits the search for a motion vector for a macroblock of a reduced image that is associated with a portion where a gradation change is moderately included in a moving image obtained by photographing a person or a landscape. Thus, it is possible to speed up the encoding process. On the other hand, when it is determined that the predicted motion vector is not accurate, the encoding unit 12 of the encoding device 10 can obtain the motion search for the macroblock of the reduced image in accordance with an instruction from the control unit 14. Encoding processing is performed using the obtained motion vector. Therefore, the encoded data obtained by the encoding device 10 shown in FIG. 1 has a higher image quality than when encoding using the predicted motion vector regardless of the likelihood of the predicted motion vector. Can be played back. That is, the encoding apparatus 10 shown in FIG. 1 uses the encoded data obtained by encoding the original moving image Vin to reproduce the second moving image having a screen size or the like different from that of the moving image Vin. In the application for generating the encoded data, it is possible to maintain both the image quality and the speed of the encoding process.

  The encoding process for the reduced image obtained by reducing each image of the moving image Vin described with reference to FIGS. 1 to 3 is performed based on the specification of the distribution destination terminal device UE from the encoded data obtained for the moving image Vin. It is an example of the transcode which produces | generates the encoding data which can reproduce | regenerate an image. The coding apparatus 10 illustrated in FIG. 1 is not limited to the example illustrated in FIG. 2, and a moving image including a reduced image reduced at various size ratios and a moving image Vin according to various frame rates are thinned out. It is possible to generate encoded data corresponding to the obtained moving image. That is, the encoding apparatus 10 illustrated in FIG. 1 performs reproduction on the encoded data generated in conformity with the specification of the terminal apparatus UE1 that is a television receiver, for example, so that the other terminal apparatus UE reproduces the encoded data. This is useful in applications where multiple types of encoded data are possible.

  In addition, the search unit 11 illustrated in FIG. 1 selects each macroblock in each image included in the moving image Vin, for example, H.264. Inter-screen prediction and intra-screen prediction may be performed for each block divided by a plurality of types of partition types (divided types) defined in the H.264 standard. In this case, the conversion unit 13 illustrated in FIG. 1 predicts each divided block from the results of inter-screen prediction and intra-screen prediction in the search unit 11 for each division type for dividing the macroblock in the reduced image. The motion vector to be performed and the prediction direction may be obtained.

  FIG. 4 shows an example of motion vector conversion by the conversion unit 13 shown in FIG. Note that among the elements shown in FIG. 4, elements equivalent to those shown in FIG. 2 are denoted by the same reference numerals and description of the elements may be omitted.

  FIG. 4A shows an example of motion vectors obtained for each of the macroblocks MB11, MB12, MB21, and MB22 shown in FIG. 2A by the search unit 11 shown in FIG. Also, FIG. 4B shows a reduced image macroblock mb11 by the conversion unit 13 shown in FIG. 1 based on the motion vectors obtained for each of the macroblocks MB11 to MB22 shown in FIG. The example of the vector estimated as a motion vector of is shown.

  Each of the solid-line arrow Vp11 and the broken-line arrow Vf11 shown in FIG. 4A indicates, for example, each of the previous image and the next image in the shooting order for the macroblock MB11. This is a motion vector obtained by inter-screen prediction. Similarly, each of the solid-line arrow Vp12 and the broken-line arrow Vf12 shown in FIG. 4A indicates the previous reference image and the next image in the shooting order for the macroblock MB12, respectively. It is a motion vector calculated | required by the inter-screen prediction used as a back side reference image. Also, each of the solid-line arrow Vp21 and the broken-line arrow Vf21 shown in FIG. 4A indicates the front reference image and the rear image for the macroblock MB21, respectively, one image before and one image in the shooting order. This is a motion vector obtained by inter-screen prediction used as a side reference image. The solid line arrow Vp22 and the broken line arrow Vf22 shown in FIG. 4 (A) respectively indicate the previous reference image and the subsequent image for the macroblock MB22 in the shooting order. This is a motion vector obtained by inter-screen prediction used as a side reference image.

  Each of the rectangles Typ1, Typ2, Typ3, and Typ4 illustrated in FIG. 4B indicates a macroblock mb11 that is divided according to each division type. In the example of FIG. 4B, the case where the macroblock mb11 is not divided is shown as a division type Typ1, and the case where it is divided into two in the horizontal direction is shown as a division type Typ2. In the example of FIG. 4B, a case where the macroblock mb11 is divided into two in the vertical direction is shown as a division type Typ3, and a case where the macroblock mb11 is divided into two in the vertical direction and the horizontal direction is shown as a division type Typ4.

  The arrows Vpc11, Vpc12, Vpc21, and Vpc22 indicated by solid lines in FIG. 4B are vectors obtained by reducing the respective motion vectors Vp11, Vp12, Vp21, and Vp22 before reduction shown in FIG. Indicates. Similarly, arrows Vfc11, Vfc12, Vfc21, and Vfc22 indicated by solid lines in FIG. 4B reduce the respective motion vectors Vf11, Vf12, Vf21, and Vf22 before reduction shown in FIG. Indicates a vector.

  In the example of FIG. 4B, as in the example of FIG. 2, the motion vector of the block divided by each division type by the conversion unit 13 is displayed in the upper left of the macroblocks before reduction associated with the block. The case where it determines based on the motion vector of the macroblock located in is shown. For example, the motion vectors Vpc11 and Vfc11 of the macroblock mb11 of the reduced image shown as the division type Typ1 in FIG. 4B are obtained by reducing each of the motion vectors Vp11 and Vf11 by the conversion unit 13. Similarly, in FIG. 4B, when the macroblock mb11 is divided by the division type Typ2, the motion vectors Vpc11 and Vfc11 of the upper block when the macroblock mb11 is divided are reduced by the conversion unit 13 respectively. Is obtained. On the other hand, in FIG. 4B, when the macroblock mb11 is divided by the division type Typ2, the motion vectors Vpc21 and Vfc21 of the lower block when the macroblock mb11 is divided are reduced by the conversion unit 13, respectively. Is obtained. Also, in FIG. 4B, the motion vectors Vpc11 and Vfc11 of the left block when the macroblock mb11 is divided by the division type Typ3 are obtained by reducing the motion vectors Vp11 and Vf11 by the conversion unit 13, respectively. It is done. On the other hand, in FIG. 4B, the motion vectors Vpc12 and Vfc12 of the right block when the macroblock mb11 is divided by the division type Typ3 are obtained by reducing each of the motion vectors Vp12 and Vf12 by the conversion unit 13. It is done. 4B, the motion vectors Vpc11 and Vfc11 of the upper left block when the macro block mb11 is divided by the division type Typ4 are obtained by reducing the motion vectors Vp11 and Vf11 by the conversion unit 13, respectively. It is done. In FIG. 4B, the motion vectors Vpc12 and Vfc12 of the upper right block when the macro block mb11 is divided by the division type Typ4 are obtained by reducing each of the motion vectors Vp12 and Vf12 by the conversion unit 13. It is done. In FIG. 4B, the motion vectors Vpc21 and Vfc21 of the lower left block when the macroblock mb11 is divided by the division type Typ4 are obtained by reducing each of the motion vectors Vp21 and Vf21 by the conversion unit 13. It is done. In FIG. 4B, the motion vector Vpc22, Vfc22 of the lower right block when the macroblock mb11 is divided by the division type Typ4 is reduced by the conversion unit 13 by reducing the motion vectors Vp22, Vf22, respectively. Desired.

  Further, as described with reference to FIG. 2, the conversion unit 13 performs the prediction image indicated by the motion vector obtained for each block obtained by dividing the macroblock mb11 with each of the division types Typ1 to Typ4 illustrated in FIG. The sum of prediction errors with each block is obtained as the second cost. As described with reference to FIG. 4, the conversion unit 13 can obtain the second cost for each macroblock of the reduced image for each inter-screen prediction mode having a different division type.

  FIG. 5 shows a conversion example of the prediction direction by the conversion unit 13 shown in FIG. Note that among the elements shown in FIG. 5, elements equivalent to those shown in FIGS. 2 and 4 are denoted by the same reference numerals and description of the elements may be omitted.

  FIG. 5A shows a prediction direction obtained by performing intra prediction for each of the macroblocks MB11, MB12, MB21, and MB22 shown in FIG. 2A by the search unit 11 shown in FIG. An example is shown. FIG. 5B shows a reduced image macroblock mb11 by the conversion unit 13 shown in FIG. 1 based on the prediction directions obtained for each of the macroblocks MB11 to MB22 shown in FIG. An example of a predicted direction as the predicted direction is shown.

  The broken arrows Vr11, Vr12, Vr21, and Vr22 shown in FIG. 5A indicate prediction errors in the intra prediction for each of the macroblocks MB11, MB12, MB21, and MB22 by the search unit 11 as compared with other directions. This indicates the prediction direction in which there are few. By the search unit 11, When the intra prediction specified in the H.264 standard is performed, each of the prediction directions Vr11, Vr12, Vr21, and Vr22 is H.264. One of nine directions defined by the H.264 standard.

  In the example of FIG. 5B, the prediction direction of the block obtained by dividing the macroblock mb11 by each division type is obtained for the macroblock located at the upper left among the macroblocks before reduction associated with the block. In this case, the predicted direction is set.

  For example, in the example shown as the division type Typ1 in FIG. 5B, the direction obtained as the prediction direction of the macroblock mb11 by the conversion unit 13 is the prediction obtained for the macroblock MB11 shown in FIG. The direction is Vr11. Similarly, for the upper block when the macroblock mb11 is divided by the division type Typ2 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB11 shown in FIG. 5A. Is the predicted direction Vr11 obtained for. On the other hand, for the lower block when the macroblock mb11 is divided by the division type Typ2 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB21 shown in FIG. Is the predicted direction Vr21 obtained for. Further, for the left block when the macroblock mb11 is divided by the division type Typ3 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB11 shown in FIG. The obtained prediction direction Vr11. On the other hand, for the right block when the macroblock mb11 is divided by the division type Typ3 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB12 shown in FIG. The obtained prediction direction Vr12 is obtained. Then, for the upper left block when the macroblock mb11 is divided with the division type Typ4 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB11 shown in FIG. The obtained prediction direction Vr11. Further, for the upper right block when the macro block mb11 is divided by the division type Typ4 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macro block MB12 shown in FIG. The obtained prediction direction Vr12 is obtained. Further, for the lower left block when the macroblock mb11 is divided by the division type Typ4 in FIG. 5B, the direction obtained as the prediction direction by the conversion unit 13 is the macroblock MB21 shown in FIG. The obtained prediction direction Vr21 is obtained. The direction obtained as the prediction direction by the conversion unit 13 for the lower right block when the macroblock mb11 is divided by the division type Typ4 in FIG. 5B is the macroblock MB22 shown in FIG. 5A. Is the predicted direction Vr22 obtained for.

  Further, the conversion unit 13 calculates an expected prediction error when the macroblock mb11 is intra-coded using the predicted direction Vr11 predicted for the macroblock mb11 shown as the division type Typ1 in FIG. The second cost shown is obtained as follows. For example, the conversion unit 13 receives from the search unit 11 the prediction error obtained for the prediction direction Vr11 when performing intra-screen prediction for each of the macroblocks MB11 to MB22, and applies a coefficient corresponding to the reduction rate to the received prediction error. The second cost is obtained by multiplying and adding each other. Here, the prediction error obtained for the prediction direction Vr11 when the intra-frame prediction is performed for each of the macroblocks MB11 to MB22 is an example of the first cost.

  Similarly, the conversion unit 13 obtains an expected prediction error for each block obtained by dividing the macroblock mb11 by each of the division types Typ2 to Typ4, and from the obtained prediction error, the conversion unit 13 obtains a prediction error for each division type Typ2 to Typ4. 2 costs can be determined. That is, the conversion unit 13 can determine the second cost for each macroblock of the reduced image from the first cost obtained for the macroblock before reduction for each intra-screen coding mode with different division types.

  As described with reference to FIGS. 4 and 5, the conversion unit 13 illustrated in FIG. The second cost can be obtained for each case where the macroblock of the reduced image is divided by each of the division types defined in the H.264 standard and the inter-screen prediction and the intra-screen prediction are performed. That is, the conversion unit 13 predicts a prediction error in each mode for each of a plurality of modes indicated by a combination of a prediction method selected from inter-screen prediction and intra-screen prediction and a division type used in the selected prediction method. 2nd cost which shows can be calculated | required.

  And when the 2nd cost calculated | required about any of several modes is below a 1st threshold value, the control part 14 is a motion vector or prediction obtained about the mode by which the 2nd cost was below the 1st threshold value. You may make the encoding part 12 perform the encoding process using a direction. Thereby, in the encoding part 12, for example, H.264. Even when encoding processing conforming to the H.264 standard is performed, the time required for encoding processing can be shortened while maintaining the quality of a moving image reproduced from encoded data.

  By the way, for example, H.H. A rough correlation between the amount of encoded data of each mode obtained when encoding a moving image by encoding processing according to the H.264 standard and the second cost obtained for each mode by the conversion unit 13 There is. This is because the amount of final encoded data is the amount of information obtained by quantizing the coefficient obtained by orthogonally transforming the prediction error, and the amount of information obtained by entropy coding the information indicating the motion vector and the prediction direction. This is because the amount of information of the former is larger than that of the latter. However, when the difference in the second cost obtained for each of the plurality of modes is small, the final encoded data of the final encoded data depends on the amount of information obtained by entropy encoding information such as motion vectors. The mode that minimizes the amount may be reversed.

  6 and 7, a method for determining whether to adopt a motion vector or a prediction direction obtained by transcoding in consideration of the possibility that the final amount of encoded data is reversed between a plurality of modes. Is explained.

  FIG. 6 shows another embodiment of the encoding device. 6 that are equivalent to the components shown in FIG. 1 are denoted by the same reference numerals and description of the components may be omitted.

  The encoding apparatus 10a illustrated in FIG. 6 includes a search unit 11a, an encoding unit 12a, a conversion unit 13a, and a control unit 14a. The search unit 11a is a component corresponding to the search unit 11 illustrated in FIG. 1 and receives a moving image Vin to be encoded from the moving image database DB. Similarly, the encoding unit 12a is a component corresponding to the encoding unit 12 illustrated in FIG. 1, and receives a moving image Vin to be encoded from the moving image database DB. In addition, the encoding unit 12a, according to an instruction from the control unit 14a, generates encoded data C1 to Cn that can be reproduced from the received moving image Vin under n types of encoding conditions having different screen sizes, frame rates, and the like. Generate. The encoded data C1 to Cn generated by the encoding unit 12a is distributed to, for example, the terminal devices UE1 to UE4 illustrated in FIG. 1 via the distribution device DT.

  The search unit 11a is, for example, H.264. In accordance with the H.264 standard or the like, for each mode described in FIGS. 4 and 5, first information indicating the motion vector or prediction direction of each macroblock in each image included in the moving image Vin, and a first cost indicating a prediction error, Ask for. The first information and the first cost obtained for each mode by the search unit 11a are passed to the conversion unit 13a.

  The conversion unit 13a is a component corresponding to the conversion unit 13 illustrated in FIG. Based on the first information received from the search unit 11a, the conversion unit 13a performs each mode described in FIG. 4 and FIG. 5 for each macroblock in the image in which the size and the like are converted according to each of the n types of encoding conditions. To obtain second information indicating a motion vector or a prediction direction expected in step (b). Further, the conversion unit 13a is described in FIG. 4 and FIG. 5 for each macroblock in the image whose size and the like are converted according to each encoding condition based on the first cost received from the search unit 11a. The second cost indicating the prediction error expected in each mode is obtained.

  The control unit 14 a illustrated in FIG. 6 is a component corresponding to the control unit 14 illustrated in FIG. 1, and includes a selection unit 141 and a suppression unit 142.

  For example, the selection unit 141 receives a first threshold (Th1) set in advance corresponding to each encoding condition from the distribution device DT. In addition, the selection unit 141 compares each received first threshold Th1 with the second cost obtained for each mode by the conversion unit 13a for the macroblock of the image whose size or the like is converted according to the same encoding condition. . For example, the selection unit 141 is obtained for the first threshold Th1 set according to the screen size of the terminal device UE3 such as a smart phone, and the macroblock of the reduced image reduced to the screen size of the terminal device UE3. The second cost of each mode is compared. Then, when the second cost obtained by the conversion unit 13a for any mode is equal to or less than the first threshold Th1, the selection unit 141 is obtained in the mode in which the second cost is equal to or less than the first threshold Th1. The second information indicating the motion vector or the prediction direction is selected. For example, when each of the second costs obtained by the conversion unit 13a for the modes in which inter-screen prediction is performed for each of the division types Typ1 and Typ4 is equal to or less than the first threshold Th1, the selection unit 141 may The second information indicating the motion vector at is selected. The second information selected by the selection unit 141 is passed to the suppression unit 142 together with the corresponding second cost.

  In the second cost obtained by the conversion unit 13, when the suppression unit 142 selects at least one second information by using the first threshold set for each encoding condition, A difference between two second costs smaller than the second cost is obtained. For example, when the selection unit 141 selects the second information obtained for the mode in which inter-screen prediction is performed for each of the division types Typ1 and Typ4, the second information obtained corresponding to the two selected modes is selected. The cost is two second costs that are smaller than the other second costs. In this case, the suppression unit 142 obtains the difference between the two second costs obtained for the modes in which inter-screen prediction is performed for each of the division types Typ1 and Typ4. And the suppression part 142 passes the 2nd information selected by the selection part 141 to the encoding part 12a, when the difference of the calculated | required 2nd cost is larger than the 2nd threshold value (Th2) preset. In this case, the encoding unit 12a performs an encoding process using the motion vector or the prediction direction indicated by the second information. On the other hand, when the obtained second cost difference is equal to or smaller than the second threshold Th2, the suppression unit 142 discards the second information selected by the selection unit 141 without passing it to the encoding unit 12a, for example. The use by the encoding unit 12a is suppressed.

  For example, the second threshold Th2 used in the suppression unit 142 is preferably set in advance to a value of the degree of variation in the code amount obtained when entropy coding is performed on information indicating a motion vector or a prediction direction. The preset second threshold Th2 may be held in a storage unit such as a memory included in the control unit 14a.

  FIG. 7 shows an example of the difference in the second cost compared with the second threshold Th2 by the suppression unit 142 shown in FIG. FIG. 7A shows a case where the difference between the second costs corresponding to the two modes selected by the selection unit 141 shown in FIG. 6 is larger than the second threshold Th2. FIG. 7B illustrates a case where the difference between the second costs corresponding to the two modes selected by the selection unit 141 illustrated in FIG. 6 is equal to or less than the second threshold Th2. In the bar graphs shown in FIGS. 7A and 7B, the horizontal axis md indicates the mode in which the second information is selected by the selection unit 141, and the vertical axis Ct indicates the conversion unit 13a for the selected mode. The size of the second cost obtained in the above is shown.

  Each of the white rectangles Cpa1 and Cpa4 illustrated in FIG. 7A represents an example of the second cost obtained by the conversion unit 13a for the mode in which inter-screen prediction is performed with each of the division types Typ1 and Typ4, for example. Each of the shaded rectangles Cta1 and Cta4 shown in FIG. 7A is a code using the motion vector obtained by the conversion unit 13a for the mode in which inter-screen prediction is performed with each of the division types Typ1 and Typ4. The example of the code amount produced | generated by digitization is shown. In FIG. 7A, an arrow d indicates a difference between the second cost Cpa1 in the mode in which inter-screen prediction is performed with the division type Typ1 and the second cost Cpa4 in the mode in which inter-screen prediction is performed with the division type Typ4.

  The code amount Cta1 is the amount of information indicating a prediction error when the motion vector obtained by the conversion unit 13a is used for the mode in which inter-screen prediction is performed with the division type Typ1, and the amount of information obtained by entropy encoding the motion vector and the like. It is indicated by the sum of Similarly, the code amount Cta4 is obtained by entropy encoding the amount of information indicating the prediction error when using the motion vector obtained by the conversion unit 13a in the mode in which inter-screen prediction is performed with the division type Typ4, the motion vector, etc. It is shown as the sum of the amount of information.

  Here, the prediction error when the motion vector obtained by the conversion unit 13a is used for the mode in which the inter-screen prediction is performed in each of the division types Typ1 and Typ4 is the second cost obtained by the conversion unit 13a for the same mode. Is almost equivalent. On the other hand, the amount of encoded information obtained by entropy encoding the motion vector and the like obtained by the conversion unit 13a for the modes in which inter-screen prediction is performed in each of the division types Typ1 and Typ4 is the second cost. Regardless, it varies within a predetermined range.

  In the example of FIG. 7A, the difference d between the second cost Cpa1 and the second cost Cpa4 is based on the second threshold value Th2 set to the same degree as the variation in the amount of information indicating the encoded motion vector or the like. Is also big. For this reason, the magnitude relationship of the second costs Cpa1, Cpa4 is the code amount Cta1, Cta4 regardless of the difference in the amount of encoded information indicating the motion vector or the like in the inter-frame prediction mode in each of the division types Typ1, Typ4. Is maintained in a large and small relationship.

  Therefore, when the difference between the second costs corresponding to the two modes selected by the selection unit 141 is larger than the second threshold Th2, the mode in which the second cost is smaller than the other modes uses the other mode. The best mode in which encoding can be performed with a smaller amount of information than encoding is shown. That is, in this case, by causing the encoding unit 12a to execute the encoding process using the motion vector indicated by the second information obtained by the selection by the selection unit 141, the image quality can be maintained and the encoding process can be performed at high speed. In addition, the amount of generated code can be further reduced.

  On the other hand, each of the white rectangles Cpb1 and Cpb4 shown in FIG. 7B is another example of the second cost obtained by the conversion unit 13a for the mode in which inter-screen prediction is performed with each of the division types Typ1 and Typ4. Indicates. Also, each of the shaded rectangles Ctb1 and Ctb4 shown in FIG. 7B is a code using the motion vector obtained by the conversion unit 13a for the mode in which inter-screen prediction is performed with each of the division types Typ1 and Typ4. Another example of the code amount generated by the conversion is shown. In FIG. 7B, the arrow d indicates the difference between the second cost Cpb1 in the mode in which inter-screen prediction is performed with the division type Typ1 and the second cost Cpb4 in the mode in which inter-screen prediction is performed with the division type Typ4.

  In the example of FIG. 7B, the difference d between the second cost Cpb1 and the second cost Cpb4 is larger than the second threshold Th2 set to be approximately the same as the variation in the amount of information obtained by entropy encoding motion vectors and the like. small. For this reason, the code amount Ctb4 is larger than the code amount Ctb1 when the code amount such as the motion vector of the mode using the division type Typ4 is larger than the code amount such as the motion vector of the mode using the division type Typ1. There is. That is, when the difference between the second costs obtained for a plurality of modes is smaller than the second threshold Th2, the mode in which the second cost is the smallest is the best mode even if the second cost is equal to or less than the first threshold. Not necessarily. Therefore, in this case, the suppression unit 142 illustrated in FIG. 6 discards the second information selected by the selection unit 141, and causes the encoding unit 12a to perform a search for a motion vector or the like to indicate the parameter. The final code amount including the amount of encoded information can be reduced.

  That is, the control unit 14 having the suppression unit 142 illustrated in FIG. 6 has encoded data generated by the encoding unit 12a as compared to a case where variation in the amount of encoded information indicating parameters such as motion vectors is not considered. The amount of can be reduced. Therefore, the encoding apparatus 10a shown in FIG. 6 is useful in applications where it is desired to suppress the amount of encoded data, for example, when there is a limit to the band that can be used for transmission of encoded data. is there.

  The encoding device 10 of the present disclosure shown in FIGS. 1 and 6 can be realized using a computer device or the like.

  FIG. 8 illustrates an example of a hardware configuration of the encoding device 10a illustrated in FIG. 8 that are the same as those shown in FIG. 6 are given the same reference numerals, and descriptions thereof are omitted.

  The computer device 20 includes a processor 21, a memory 22, a storage device 23, a network interface 24, a video interface 25, an encoding engine 26, and an optical drive device 27. The processor 21, memory 22, storage device 23, network interface 24, video interface 25, encoding engine 26, and optical drive device 27 shown in FIG. 8 are connected to each other via a bus. . Further, the processor 21, the memory 22, the storage device 23, the video interface 25, and the encoding engine 26 are included in the encoding device 10a. On the other hand, the processor 21, the memory 22, the storage device 23, and the network interface 24 are included in the distribution device DT. That is, the example illustrated in FIG. 8 illustrates a hardware configuration in the case where the encoding device 10a and the distribution device DT are one device.

  The computer apparatus 20 shown in FIG. 8 is connected to a network NW such as the Internet via a network interface 24, and information can be exchanged between the server apparatus SV and the terminal apparatus UE connected to the network NW. is there.

  The video interface 25 shown in FIG. 8 is connected to, for example, the video camera CAM, and acquires a moving image shot by the video camera CAM as a moving image Vin to be encoded by the encoding device 10a. To do.

  The encoding engine 26 is a component corresponding to the search unit 11a and the encoding unit 12a illustrated in FIG. In response to an instruction from the processor 21, the encoding engine 26 is, for example, H.264. As a process for encoding the moving image Vin in accordance with the H.264 standard or the like, a process including a motion vector search by inter-screen prediction and a prediction direction search by intra-screen prediction is executed. Note that the functions of the search unit 11a and the encoding unit 12a illustrated in FIG. 6 may be realized by the processor 21 executing a program.

  Further, the optical drive device 27 shown in FIG. 8 can be mounted with a removable disk 28 such as an optical disk, and reads and records information recorded on the mounted removable disk 28.

  The memory 22 illustrated in FIG. 8 stores an application program for the processor 21 to execute an encoding process illustrated in FIG. Note that the application program for executing the encoding process shown in FIG. 9 can be recorded and distributed on a removable disk 28 such as an optical disk, for example. Then, the application program for executing the encoding process may be stored in the memory 22 and the storage device 23 by attaching the removable disk 28 to the optical drive device 27 and performing the reading process. Alternatively, an application program for executing the encoding process may be downloaded via the network interface 24, and the downloaded application program may be read into the memory 22 and the storage device 23.

  Then, the processor 21 performs the functions of the conversion unit 13a and the control unit 14a illustrated in FIG. 6 by executing the application program stored in the memory 22.

  That is, the encoding device 10a illustrated in FIG. 6 is realized by the cooperation of the processor 21, the memory 22, the storage device 23, the video interface 25, and the encoding engine 26 included in the computer device 20 illustrated in FIG. can do.

  FIG. 9 shows the operation of the encoding device 10a shown in FIG. Each process of step S311 to step S322 illustrated in FIG. 9 is an example of a process included in the application program for the encoding process. In addition, each process of step S311 to step S322 is executed by the processor 21 shown in FIG. 8 and the encoding engine 26 that operates according to an instruction from the processor 21.

  In step S311, the processor 21 passes the moving image Vin received from the video interface 25 to the encoding engine 26, and for each macroblock in each image of the moving image Vin, between the screens of the modes described in FIGS. Prediction and in-screen prediction processing are executed. For example, each time the processor 21 receives one or more GOPs (Group of Picture) from the video camera CAM via the video interface 25, the processor 21 executes the process of step S311. In the process of step S311, the processor 21 stores, for example, in the storage device 23, information indicating the motion vector and prediction error of each mode obtained by performing the inter-screen prediction process in the encoding engine 26, It may be used in subsequent processing. Similarly, the processor 21 accumulates information indicating the prediction direction and prediction error of each mode obtained by performing the intra-screen prediction processing in the encoding engine 26 in the storage device 23 and uses it in the subsequent processing. Also good.

  In step S312, the processor 21 determines the size and frame rate of the moving image Vin received in the process of step S311 for each of n types of encoding conditions set in advance corresponding to the terminal device UE that is the distribution destination. Generate converted video. For example, the processor 21 sequentially selects n types of encoding conditions, and thins out images from the moving image Vin in accordance with the frame rate specified by the selected encoding conditions. Then, the processor 21 generates a second moving image by reducing each of the images remaining after the thinning out at a reduction rate specified by the selected encoding condition.

  In step S313, as described with reference to FIG. 2, the processor 21 includes each macroblock in the reduced image included in the second moving image generated by the process in step S312 included in the moving image Vin before reduction. Associate macroblocks in the image.

  In step S314, the processor 21 indicates, for each macroblock of the reduced image, a motion vector or a prediction direction predicted in each mode of inter-screen prediction and intra-screen prediction, as described with reference to FIGS. The second information indicating the second information and the prediction error is obtained.

  In step S315, the processor 21 determines whether or not there is a mode in which the second cost equal to or less than the first threshold Th1 described in FIG. 6 is obtained in the process of step S314.

  For at least one mode, when the second cost equal to or lower than the first threshold is obtained in the process of step S314 (Yes determination in step S315), the processor 21 proceeds to step S316.

  In step S316, the processor 21 obtains a difference between two second costs having a smaller value than the other second costs among the second costs obtained for each mode in the process of step S314, and obtains the obtained difference. Is less than or equal to the second threshold described with reference to FIG.

  As in the example of FIG. 7A, when the difference between two second costs having a value smaller than the other second costs is larger than the second threshold Th2 (No determination in step S316 (NO)). The processor 21 executes the process of step S317.

  In step S317, for example, the processor 21 passes the second information corresponding to the mode in which the smallest second cost is obtained in the process of step S314 to the encoding engine 26, and performs encoding using the parameter indicated by the second information. Is executed by the encoding engine 26. For example, in the example of FIG. 7A, the processor 21 passes the motion vector obtained by the inter-frame prediction using the division type Typ4 and the second information indicating the reference image to the encoding engine 26. In this case, the encoding engine 26 omits the inter-screen prediction process and the intra-screen prediction process for the macroblock of the reduced image, and uses the motion vector and the reference image indicated by the second information received from the processor 21. Execute the conversion process.

  On the other hand, when the second cost having a value equal to or less than the first threshold value is not obtained in the process of step S314 (No determination in step S315 (NO)), the processor 21 executes the processes of step S318 and step S319. . Further, when it is determined in the process of step S316 that the difference between the two second costs having a value smaller than the other second costs is equal to or less than the second threshold Th2 (Yes determination in step S316 (YES)), The processor 21 executes the processes of step S318 and step S319.

  In step S318, the processor 21 causes the encoding engine 26 to perform inter-screen prediction and intra-screen prediction for the macroblock of the reduced image.

  In step S319, the processor 21 causes the encoding engine 26 to perform encoding using the information obtained by the inter-screen prediction and the intra-screen prediction in step S318.

  After the process of step S316 described above is completed, or after the processes of step S318 and step S319 are completed, the processor 21 executes the process of step S320.

  In step S320, the processor 21 determines whether or not the encoding process described in steps S314 to S319 has been executed for all macroblocks of the reduced image.

  When there is a macroblock for which encoding has not been completed (No at Step S320 (NO)), the processor 21 returns to Step S314 and starts encoding processing for a new macroblock.

  On the other hand, when the encoding process for all macroblocks included in the reduced image is completed (Yes in step S320), the processor 21 proceeds to the process in step S321.

  In step S321, the processor 21 determines whether or not the encoding has been completed for all the reduced images included in the moving image generated by the conversion based on the encoding condition in the process of step S312.

  When there is a reduced image for which encoding has not been completed (No in step S321 (NO)), the processor 21 returns to step S313 and starts encoding processing for a new reduced image.

  On the other hand, when the encoding process has been completed for all the reduced images included in the moving image generated by the conversion based on the encoding condition (Yes in step S321 (YES)), the processor 21 performs the process in step S322. Proceed to

  In step S322, the processor 21 determines whether or not the encoding process has been completed for all moving images converted under each of n types of preset encoding conditions.

  When there is an encoding condition for which encoding has not been completed (No in step S321 (NO)), the processor 21 returns to step S312 and performs an encoding process for a moving image converted under the new encoding condition. To start.

  On the other hand, when the encoding process for all the moving images generated by the conversion based on each of the n types of encoding conditions is completed (Yes in step S322 (YES)), the processor 21 ends the process. .

  By executing the processing described above, the encoding device 10a illustrated in FIG. 8 converts, for example, a moving image Vin obtained by shooting with the video camera CAM into a moving image converted according to each type of encoding condition. Corresponding encoded data can be generated in near real time. Accordingly, the distribution device DT illustrated in FIG. 8 converts the moving image Vin obtained by shooting by the video camera CAM or the moving image Vin according to the screen size of the terminal device UE to each terminal device. Distribution to the UE in near real time is possible.

  From the above detailed description, features and advantages of the embodiment will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and changes. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate modifications and equivalents included in the scope disclosed in the embodiments can be used.

DESCRIPTION OF SYMBOLS 10,10a ... Encoding apparatus; 11, 11a ... Search part; 12, 12a ... Encoding part; 13, 13a ... Conversion part; 14, 14a ... Control part; 141 ... Selection part; 142 ... Deterrence part; 21; Processor; 22 ... Memory; 23 ... Storage device; 24 ... Network interface; 25 ... Video interface; 26 ... Encoding engine; 27 ... Optical drive device; 28 ... Removable disk; DB ... Video database; Distribution device; NW ... Network; UE, UE1, UE2, UE3, UE4 ... Terminal device; SV ... Server device; CAM ... Video camera

Claims (5)

By performing at least one of inter-screen prediction and intra-screen prediction for each of the plurality of first blocks obtained by dividing each of the plurality of first images included in the first moving image, the motion vector and the prediction direction A search unit that searches for at least one of
Each of a plurality of second images included in a second moving image that is generated from the first moving image and has at least one of a frame rate and a screen size different from that of the first moving image is defined as the second image. An encoding unit for encoding each second block obtained by the division;
First information indicating at least one of a motion vector and a prediction direction obtained by the search unit, and a first cost indicating a prediction error that is a difference between the predicted image indicated by the first information and the first block A first indicating at least one of a motion vector and a prediction direction expected to be obtained when performing at least one of inter-frame prediction and intra-screen prediction for each of the second blocks included in each of the second images. A conversion unit for obtaining each of two information and a second cost indicating a prediction error predicted between the prediction image indicated by the second information and the second block;
When encoding the second block for which a second cost equal to or lower than a predetermined first threshold is encoded by the conversion unit, the encoding unit is encoded using the second information obtained by the conversion unit. An encoding device comprising: a control unit. The encoding device according to claim 1, wherein
In the search using at least one of inter-frame prediction and intra-screen prediction, the conversion unit is configured to divide the second block into at least one third block in each of a plurality of different modes, and the motion vector and the Second information including third information indicating at least one of the prediction directions, a second cost indicated by a sum of third costs indicating a difference between the location indicated by the third information and the third block; Seeking
The control unit has a second difference between the second costs obtained by the conversion unit in each of the plurality of modes, the difference between two second costs being smaller than the other second costs is smaller than the first threshold value. An encoding apparatus comprising: a suppression unit that suppresses the use of the second information by the encoding unit when the threshold value is 2 or less. The encoding device according to claim 2,
The said suppression part uses the value which shows dispersion | distribution of the code amount produced | generated when the said 2nd information is encoded by the entropy encoding in the said encoding part as said 2nd threshold value. . By performing at least one of inter-screen prediction and intra-screen prediction for each of the plurality of first blocks obtained by dividing each of the plurality of first images included in the first moving image, the motion vector and the prediction direction Explore at least one of
Each of a plurality of second images included in a second moving image that is generated from the first moving image and has at least one of a frame rate and a screen size different from that of the first moving image is defined as the second image. When encoding every second block obtained by dividing,
First information indicating at least one of a motion vector and a prediction direction obtained by the search, and a first cost indicating a prediction error that is a difference between the predicted image indicated by the first information and the first block, respectively. To the second block indicating at least one of a motion vector and a prediction direction expected to be obtained when at least one of inter-frame prediction and intra-screen prediction is performed for each of the second blocks included in each of the second images. Each of the information and a second cost indicating a prediction error predicted between the predicted image indicated by the second information and the second block;
An encoding method comprising: encoding the second block for which a second cost equal to or less than a predetermined first threshold is obtained using second information obtained for the second block. By performing at least one of inter-screen prediction and intra-screen prediction for each of the plurality of first blocks obtained by dividing each of the plurality of first images included in the first moving image, the motion vector and the prediction direction Explore at least one of
Each of a plurality of second images included in a second moving image that is generated from the first moving image and has at least one of a frame rate and a screen size different from that of the first moving image is defined as the second image. When encoding every second block obtained by dividing,
First information indicating at least one of a motion vector and a prediction direction obtained by the search, and a first cost indicating a prediction error that is a difference between the predicted image indicated by the first information and the first block, respectively. To the second block indicating at least one of a motion vector and a prediction direction expected to be obtained when at least one of inter-frame prediction and intra-screen prediction is performed for each of the second blocks included in each of the second images. Each of the information and a second cost indicating a prediction error predicted between the predicted image indicated by the second information and the second block;
The second block for which the second cost equal to or lower than a predetermined first threshold is obtained is encoded using the second information obtained for the second block.
An encoding program for causing a computer to execute processing.

Images