JP6825699B2 - Moving image processing device, moving image processing method, and moving image processing program - Google Patents

Description

The present invention relates to a technique of dividing an image into a plurality of pixel blocks and performing coding processing or decoding processing for each divided pixel block.

H.264 / MPEG (Moving Picture Experts Group) -4 AVC (Advanced Video Coding) (hereinafter referred to as "H.264") and H.265 / HEVC (High Efficiency Video Coding) (hereinafter referred to as "H.265") ), The image is divided into a large number of pixel blocks (for example, 16x16 pixel block, 4x4 pixel block, etc.), and each pixel block is encoded and decoded.

In the coding of moving images such as H.264, the predicted image is generated based on the temporal or spatial correlation, and the information is encoded by encoding only the information indicating the difference between the input image and the predicted image. Compress. The encoder that performs such information compression processing can select the coding type that specifies the method of generating the predicted image from either inter-prediction or intra-prediction. Inter-prediction is a method of creating a prediction image based on pixels constituting neighboring frames based on temporal correlation. On the other hand, intra prediction is a method of generating a prediction image based on neighboring pixels constituting the same frame based on spatial correlation. For example, in intra-prediction based on H.264, processing is performed in raster scan order from the pixel block located at the upper left of the frame. That is, in the intra-prediction based on H.264, when predicting the pixel value of a target pixel block, for example, the encoded or decoded pixels located on the left, upper left, upper, and upper right of the target pixel block. Process based on the pixel value of the block. These pixel blocks are pixel blocks adjacent to the target pixel block that should be referred to when the target pixel block that has not yet been encoded (or decoded) is encoded (or decoded). In the present application, these pixel blocks will be hereinafter referred to as "reference adjacent blocks".

FIG. 17 is a block diagram conceptually showing the configuration of a general moving image decoding device 40 conforming to the H.264 standard. The moving image decoding device 40 includes a variable length decoding unit 410, an inverse transformation inverse quantization unit 411, a loop filter 412, a frame buffer 413, an inter prediction unit 414, and an intra prediction unit 415.

The variable-length decoding unit 410 decodes the bit stream of the input moving image. Inverse conversion The inverse quantization unit 411 reverse-quantizes the signal decoded by the variable-length decoding unit 410 and then performs inverse conversion. The moving image decoding device 40 generates a restored image by adding the signal output by the inverse transformation inverse quantization unit 411 and the predicted image described later. The loop filter 412 outputs a decoded image which is the result of removing the block distortion in the generated restored image. The frame buffer 413 holds the decoded image output from the loop filter 412.

The above-mentioned prediction image is generated by the inter prediction unit 414 or the intra prediction unit 415. The inter-prediction unit 414 generates a prediction image by using the pixels of the decoded image held in the frame buffer 413. Since the inter-prediction unit 414 does not refer to the pixels in the same frame when processing the pixel block (inter-prediction block) to be processed, there is no dependency on the processing order between the inter-prediction blocks of the same frame. .. Therefore, the inter-prediction unit 414 can process the processing for the plurality of inter-prediction blocks in high parallel regardless of their processing status (independently).

On the other hand, the intra prediction unit 415 generates a prediction image regarding a pixel block (intra prediction block) to be processed by using the pixels of the restored image included in the reference adjacent block. That is, there is a dependency related to the processing order that the intra prediction unit 415 cannot perform the processing on the intra prediction block until the processing on the reference adjacent block is completed. Therefore, even when such a dependency exists, there are increasing expectations for a technique for speeding up the coding process or the decoding process for a moving image.

As a technique related to such a technique, Patent Document 1 discloses an apparatus that performs coding processing or decoding processing in parallel in units of divided pixel blocks. This device starts processing from the pixel block at the upper left end in the frame, and manages whether or not the processing for each pixel block is completed. When the processing for a certain pixel block is completed, this device searches for the pixel block for which the processing of the pixel blocks located on the left and upper right has been completed as the pixel block to be processed next, and uses the searched pixel block as the searched pixel block. On the other hand, the processing is repeated. Since this device proceeds from the leftmost pixel block to the right, if it can be confirmed that the processing of the pixel block located in the upper right is completed, the processing of the pixel blocks located in the upper left and above is completed. The relationship can be guaranteed.

In FIG. 18, on the premise that the above-mentioned relationship is guaranteed, for example, a general moving image decoding device 40 using the technique shown in Patent Document 1 performs prediction processing on a pixel block (individual quadrangle shown in FIG. 18). It is a figure which shows an example of the order of executing in parallel. The position of the pixel block in FIG. 18 can be represented by the (x, y) coordinate values in the two-dimensional space. In FIG. 18, the number assigned to each pixel block indicates the processing order for each pixel block, and indicates that the pixel blocks to which the same number is assigned can be processed in parallel.

In the example shown in FIG. 18, first, the pixel block located at the coordinate value (1,1) representing the upper left of the frame (hereinafter referred to as the pixel block (1,1) in the present application, and the other pixel blocks are similarly referred to. The process for (referred to as) is performed. Next (second), processing is performed on the pixel block (2, 1) located to the right of the pixel block (1, 1). When the processing for the pixel block (2,1) is completed, the pixel block (3,1) located to the right of the pixel block (2,1), and the pixel blocks (1,1) and (2,1). It is possible to execute the processing for the pixel blocks (1, 2) having the reference adjacent block. As a result, the processing for the pixel blocks (3, 1) and (1, 2) is performed in parallel as the third processing. In the example shown in FIG. 18, similarly, the pixel block (m, n) (m is an arbitrary integer of 3 or more, n is an arbitrary integer of 1 or more) and the pixel block (m-2, n + 1) Is processed in parallel. However, "-" is an operator representing subtraction, and "+" is an operator representing addition.

Further, in Patent Document 2, a moving image stream encoded in macroblocks (pixel blocks) of n × n pixels (n is an arbitrary natural number) generated by dividing each image into a matrix is decoded. The device is disclosed. This device detects an inter-macro block (inter-prediction block) in the slice by analyzing information representing a slice composed of one or more macro blocks included in the input moving image stream. This device includes two or more decoding processing means for executing decoding processing in units of macroblocks. Then, this device executes the decoding process for the detected intermacroblock in the slice in parallel by the two or more decoding processing means, and then decodes the intramacroblock (intra prediction block) in the slice. Is controlled to be executed by the decoding processing means.

Japanese Unexamined Patent Publication No. 2006-129284 Japanese Patent Application Laid-Open No. 2009-038501

"Information Source Coding Department H.264 | Overview of MPEG-4 AVC Standard", [Online], [Searched on March 29, 2017], Internet <URL: http://www.soumu.go.jp/main_sosiki /joho_tsusin/policyreports/joho_tsusin/bunkakai/pdf/060720_3_1-2_sa2.pdf>

In the above-mentioned general moving image decoding processing or moving image coding processing, for example, as illustrated in FIG. 18, the pixel blocks (intra-prediction blocks) to be processed are ordered from a predetermined position (for example, upper left) in the frame. Search for. Then, when all the processing for the reference adjacent block related to the searched intra prediction block (for example, the pixel blocks located on the left, upper left, upper, and upper right of the intra prediction block) is completed, the processing for the intra prediction block is completed. To start.

In the case of performing such processing, for example, consider a case where all the reference adjacent blocks related to a certain intra prediction block are inter prediction blocks. As described above, the processing for the inter-prediction block can be processed in parallel regardless of their processing status (independently), so that the processing can be completed in a short time by being processed in high parallel. it can. Then, when the intra prediction block is located near the lower right of the frame, for example, the intra prediction block is searched for near the end in the search order. In this case, although the intra prediction block can be processed at an early timing, the timing at which the processing is actually started is delayed. That is, in general moving image decoding processing or moving image coding processing, there are cases where the start of processing for the intra prediction block that can be processed at an early timing is delayed, so that the processing speed is sufficient. I can not say. Patent Documents 1 and 2 do not specifically mention this problem. A main object of the present invention is to provide a moving image processing device or the like that solves such a problem.

The moving image processing apparatus according to one aspect of the present invention is a case where an image is divided into a plurality of pixel blocks and image processing is performed on each of the pixel blocks to perform coding processing or decoding processing on the image. A storage means for storing the execution state of the image processing for each pixel block, and a second pixel block stored in the storage means for which the image processing for the first pixel block has a dependency relationship with respect to the processing order. A determination means for determining whether or not the image processing for the first pixel block can be executed based on the execution state of the image processing, and a plurality of said firsts for which the determination means has determined that the image processing can be executed. The image processing for one pixel block is executed in parallel or pseudo-parallel, and the execution means for updating the execution state of the image processing for the first pixel block stored in the storage means is provided.

From another viewpoint of achieving the above object, the moving image processing method according to one aspect of the present invention is performed by dividing an image into a plurality of pixel blocks by an information processing apparatus and performing image processing on each of the pixel blocks. When the encoding process or the decoding process for the image is performed, the execution state of the image processing is stored in the storage means for each pixel block, and the image processing for the first pixel block stored in the storage means. Determines whether or not the image processing for the first pixel block can be executed based on the execution state of the image processing for the second pixel block having a dependency relationship with respect to the processing order, and the image processing can be executed. The image processing for the plurality of determined first pixel blocks is executed in parallel or pseudo-parallel, and the execution state of the image processing for the first pixel block stored in the storage means is updated.

Further, from the further viewpoint of achieving the above object, the moving image processing program according to one aspect of the present invention divides an image into a plurality of pixel blocks and performs image processing on each of the pixel blocks to perform the image processing. A storage control process for storing the execution state of the image processing in the storage means for each pixel block and an image for the first pixel block stored in the storage means when the coding process or the decoding process for the first pixel block is performed. A determination process for determining whether or not image processing for the first pixel block can be executed based on an execution state of image processing for the second pixel block whose processing has a dependency relationship with respect to the processing order, and the determination process. The image processing for the plurality of first pixel blocks determined to be feasible is executed in parallel or pseudo-parallel, and the image processing for the first pixel block stored in the storage means is performed. It is a program for causing a computer to execute an execution process that updates the execution status.

Further, the present invention can also be realized by a computer-readable, non-volatile recording medium in which the moving image processing program (computer program) is stored.

The present invention makes it possible to speed up the coding process or the decoding process for a moving image.

It is a block diagram which shows the structure of the moving image decoding apparatus 10 which concerns on 1st Embodiment of this invention. It is a flowchart which shows the whole operation of the moving image decoding apparatus 10 which concerns on 1st Embodiment of this invention. It is a flowchart which shows the detail of the operation which the moving image decoding apparatus 10 which concerns on 1st Embodiment of this invention performs the prediction processing using the inter prediction method for a pixel block. It is a flowchart which shows the detail of the operation which the moving image decoding apparatus 10 which concerns on 1st Embodiment of this invention performs the prediction processing using the intra prediction method for a pixel block. It is a figure which shows the comparative example of the order in which the moving image decoding apparatus 10 which concerns on 1st Embodiment of this invention, and the general moving image decoding apparatus execute the prediction processing with respect to a pixel block in parallel. It is a figure which illustrates the content of each intra prediction mode referred to by the moving image decoding apparatus 10A which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the detail of the operation which the moving image decoding apparatus 10A which concerns on 2nd Embodiment of this invention performs the prediction processing using the intra prediction method for a pixel block. It is a figure which shows the comparative example of the order in which the moving image decoding apparatus 10A which concerns on the 2nd Embodiment of this invention, and the general moving image decoding apparatus execute the prediction processing with respect to a pixel block in parallel. It is a figure which illustrates the dependency index about the pixel block updated by the moving image decoding apparatus 10B which concerns on 3rd Embodiment of this invention. It is a flowchart which shows the detail of the operation which the moving image decoding apparatus 10B which concerns on 3rd Embodiment of this invention performs the prediction processing using the intra prediction method for a pixel block. It is a block diagram which shows the structure of the moving image coding apparatus 20 which concerns on 4th Embodiment of this invention. It is a flowchart which shows the whole operation of the moving image coding apparatus 20 which concerns on 4th Embodiment of this invention. It is a flowchart which shows the detail of the operation which the moving image coding apparatus 20 which concerns on 4th Embodiment of this invention performs the prediction processing using the inter prediction method for a pixel block. A flowchart showing the details of the operation of the moving image coding device 20 according to the fourth embodiment of the present invention to perform a prediction process using an intra prediction method on a pixel block. It is a block diagram which shows the structure of the moving image processing apparatus 30 which concerns on 5th Embodiment of this invention. It is a block diagram which shows the structure of the moving image decoding apparatus 10 (10A, 10B) which concerns on each embodiment of this invention, the moving image coding apparatus 20, and the information processing apparatus 900 which can execute the moving image processing apparatus 30. It is a block diagram which shows the structure of the general moving image decoding apparatus 40. It is a figure which shows an example of the order in which a general moving image decoding apparatus 40 executes a prediction process for a pixel block in parallel.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram conceptually showing the configuration of the moving image decoding device 10 according to the first embodiment of the present invention. Note that FIG. 1 will also be referred to in the description of the moving image decoding device 10A according to the second embodiment and the moving image decoding device 10B according to the third embodiment, which will be described later.

The moving image decoding device 10 according to the present embodiment conforms to a standard such as H.264 or H.265 for an input bit stream representing a moving image encoded by a moving image coding device (not shown). It is a device that performs a decoding process and outputs a decoded image as a result of the decoding process. The moving image decoding device 10 performs decoding processing in parallel or pseudo-parallel for each pixel block (for example, 16x16 pixel block, 4x4 pixel block, etc.) included in the frame constituting the moving image. However, pseudo-parallel is parallel processing by, for example, time division. Hereinafter, in the present application, it will be referred to as "parallel" including "pseudo-parallel".

The input bitstream to be decoded by the moving image decoding device 10 encodes only information indicating the difference between the predicted image and the input image (original image) generated based on the temporal or spatial correlation. This is the information compressed by. Then, the prediction image regarding each pixel block included in the input bit stream is generated by inter-prediction (method) or intra-prediction (method). Inter-prediction is a method of generating a predicted image using pixels included in neighboring frames based on temporal correlation. Intra-prediction is a method of generating a predicted image using neighboring pixels included in the same frame based on spatial correlation.

The moving image decoding device 10 according to the present embodiment is roughly classified into an execution unit 11, a storage unit 12, and a determination unit 13.

The storage unit 12 is a storage device such as an electronic memory or a magnetic disk. The storage unit 12 stores the execution state of the inter-prediction processing (image processing) by the inter-prediction unit 114 and the intra-prediction processing (image processing) by the intra-prediction unit 115 for each pixel block. These execution states are updated by the inter-prediction unit 114 or the intra-prediction unit 115. The storage unit 12 also stores the coding type (information) output by the variable length decoding unit 110, which will be described later, in association with the pixel block. The coding type will be described later.

When the intra-prediction unit 115, which will be described later, selects a certain unprocessed pixel block (first pixel block) as the intra-prediction processing target, the determination unit 13 can execute the intra-prediction processing for the pixel block. Judge whether or not. More specifically, the determination unit 13 sequentially sets pixel blocks (second pixel blocks) located on the left, upper left, upper, and upper right of the pixel block as reference adjacent blocks. The execution state is confirmed by referring to the storage unit 12. The pixel block set as the reference adjacent block by the determination unit 13 may be a pixel block located at a position other than the left, upper left, upper, and upper right with respect to the pixel block. When the determination unit 13 has completed all the prediction processing for the reference adjacent block (that is, the dependency on the processing order from the selected pixel block to the reference adjacent block has been eliminated), the intra prediction unit 115 makes an intra. It is determined that the prediction process can be executed, and the determination result is notified to the intra prediction unit 115.

The execution unit 11 includes a variable length decoding unit 110, an inverse transformation inverse quantization unit 111, a loop filter 112, a frame buffer 113, an inter prediction unit 114, an intra prediction unit 115, and switches 116 and 117.

The variable-length decoding unit 110 outputs information including the post-quantization conversion coefficient and the coding type by performing decoding processing on the input bit stream. As for the post-quantization conversion coefficient, a well-known technique can be adopted in the technical field of the present invention, and a detailed description thereof will be omitted. The coding type is information that identifies for each pixel block whether it is encoded by inter-prediction or intra-prediction. The variable-length decoding unit 110 stores the coding type in the storage unit 12 in association with the pixel block.

Inverse conversion The inverse quantization unit 111 outputs a residual signal by inversely converting the post-quantization conversion coefficient output from the variable length decoding unit 110 after inverse quantization. Since a well-known technique can be adopted in the technical area of the present invention for the process performed by the inverse transform inverse quantization unit 111, detailed description thereof will be omitted.

The inter-prediction unit 114 refers to the coding type and the execution state of the prediction process for each pixel block stored in the storage unit 12. The inter-prediction unit 114 selects an inter-prediction block for which prediction processing has not been processed from among pixel blocks (inter-prediction blocks) whose coding type indicates "inter-prediction", and inter-predicts the selected inter-prediction block. Perform processing. More specifically, the inter-prediction unit 114 generates a prediction image representing the result of the inter-prediction processing by referring to the decoded frame (image) stored in the frame buffer 113 described later. The execution unit 11 generates a restored image by adding the generated predicted image and the residual signal output from the inverse transformation inverse quantization unit 111. The inter-prediction unit 114 updates the execution state stored in the storage unit 12 to "completed" with respect to the inter-prediction block that has completed the prediction process.

The intra prediction unit 115 refers to the coding type and the execution state of the prediction process for each pixel block stored in the storage unit 12. The intra prediction unit 115 selects an intra prediction block whose prediction process has not been processed from among the pixel blocks (intra prediction blocks) whose coding type indicates “intra prediction”.

When the determination unit 13 notifies that the intra prediction process can be executed for the selected intra prediction block, the intra prediction unit 115 starts the intra prediction process for the intra prediction block. More specifically, the intra prediction unit 115 generates a prediction image representing the result of performing the intra prediction processing by referring to the restored image relating to the adjacent reference block. However, it is assumed that the restored image relating to the adjacent reference block is stored by the intra prediction unit 115, for example, in the storage unit 12. The execution unit 11 generates a restored image by adding the generated predicted image and the residual signal output from the inverse transformation inverse quantization unit 111. The intra prediction unit 115 updates the execution state stored in the storage unit 12 to "completed" with respect to the intra prediction block that has completed the prediction process.

The execution unit 11 switches the above-mentioned inter-prediction processing and intra-prediction by switching the switches 116 and 117 for switching the input / output of the inter-prediction unit 114 and the intra-prediction unit 115 according to the coding type of the pixel block to be processed. It is assumed that the processing can be switched as appropriate. As for this switching, a well-known technique can be adopted in the technical field of the present invention, and a detailed description thereof will be omitted. The loop filter 112 generates a decoded image in which block distortion is removed from the restored image generated by the execution unit 11, and outputs the generated decoded image to the outside. The loop filter 112 stores the generated decoded image in the frame buffer 113.

Next, the operation (processing) of the moving image decoding device 10 according to the present embodiment will be described in detail with reference to the flowcharts of FIGS. 2 to 4.

FIG. 2 is a flowchart showing the entire operation of the moving image decoding device 10.

The variable-length decoding unit 110 in the execution unit 11 outputs information including the post-quantization conversion coefficient and the coding type by performing decoding processing on the input bit stream (step S101). The inverse conversion inverse quantization unit 111 in the execution unit 11 outputs a residual signal by inversely converting the post-quantization conversion coefficient output from the variable length decoding unit 110 after inverse quantization (step S102).

The execution unit 11 generates a restored image by performing inter-prediction processing on a pixel block (inter-prediction block) whose coding type is inter-prediction (step S103). The execution unit 11 generates a restored image by performing an intra prediction process on a pixel block (intra prediction block) whose coding type is intra prediction (step S104). The loop filter 112 in the execution unit 11 generates a decoded image in which the block distortion is removed from the generated restored image, outputs the generated decoded image to the outside, and stores it in the frame buffer 113 (step S105). The process ends.

FIG. 3 is a flowchart showing the details of the operation of the moving image decoding device 10 for performing the prediction processing using the inter-prediction method on the pixel block. That is, FIG. 3 is a flowchart showing the details of step S103 in the flowchart shown in FIG.

The inter-prediction unit 114 in the execution unit 11 selects an unprocessed inter-prediction block by referring to the execution state of the prediction process for each pixel block stored in the storage unit 12 (step S201). The inter-prediction unit 114 performs inter-prediction processing on the selected inter-prediction block (step S202).

The inter-prediction unit 114 updates the execution state stored in the storage unit 12 to "completed" with respect to the inter-prediction block that has completed the prediction process (step S203). The inter-prediction unit 114 confirms whether or not there is an unprocessed inter-prediction block by referring to the storage unit 12 (step S204).

If there is an unprocessed inter-prediction block (Yes in step S205), the process returns to step S201. If there is no unprocessed inter-prediction block (No in step S205), the processing in step S103 ends.

FIG. 4 is a flowchart showing the details of the operation of the moving image decoding device 10 for performing the prediction processing using the intra prediction method on the pixel block. That is, FIG. 4 is a flowchart showing the details of step S104 in the flowchart shown in FIG.

The intra prediction unit 115 in the execution unit 11 selects an unprocessed intra prediction block by referring to the execution state of the prediction processing for each pixel block stored in the storage unit 12 (step S301). The determination unit 13 sequentially sets pixel blocks located on the left, upper left, upper, and upper right of the intra prediction block selected by the intra prediction unit 115 as reference adjacent blocks, and refers to the storage unit 12. , It is confirmed whether or not the prediction process for the reference adjacent block is completed (step S302).

If the prediction process for the reference adjacent block is not completed (No in step S303), the process returns to step S301. When the prediction process for the reference adjacent block is completed (Yes in step S303), the determination unit 13 performs the loop process from step S302 for the next reference adjacent block, and the process for all the reference adjacent blocks is completed. If so, the process proceeds to step S305 (step S304).

The determination unit 13 determines that the intra prediction process for the intra prediction block selected by the intra prediction unit 115 can be executed, and notifies the intra prediction unit 115 of the determination result (step S306). The intra prediction unit 115 performs an intra prediction process on the selected intra prediction block (step S306).

The intra-prediction unit 115 updates the execution state stored in the storage unit 12 to "completed" with respect to the intra-prediction block for which the prediction process has been completed (step S307). The intra prediction unit 115 confirms whether or not there is an unprocessed intra prediction block by referring to the storage unit 12 (step S308).

If there is an unprocessed intra prediction block (Yes in step S309), the process returns to step S301. If there is no unprocessed intra prediction block (No in step S309), the processing in step S104 ends.

The moving image decoding device 10 according to the present embodiment can speed up the decoding process for moving images. The reason is that the moving image decoding device 10 executes the prediction process (image processing) for each pixel block when the first pixel block has a processing order dependency on the second pixel block. This is because it is possible to detect that the dependency has been resolved at an early timing by managing the above, and as a result, it is possible to accelerate the start of the prediction process for the first pixel block.

The effects realized by the moving image decoding apparatus 10 according to the present embodiment will be described in detail below.

In the general moving image decoding process (the same applies to the moving image coding process), the intra prediction blocks to be processed are searched in order from, for example, the upper left of the frame. Then, when all the processing for the reference adjacent block related to the searched intra prediction block is completed, the processing for the intra prediction block is started. In the case of performing such processing, for example, when all the reference adjacent blocks related to a certain intra prediction block are inter prediction blocks and are located near the lower right of the frame, the intra prediction block can be processed at an early timing. Nevertheless, the timing at which the processing is actually started is delayed. That is, in general moving image decoding processing or moving image coding processing, there is a case where the start of processing for the intra prediction block that can be processed at an early timing is delayed, so that there is a problem that the speed of processing is not sufficient. is there.

To deal with such a problem, the moving image decoding device 10 according to the present embodiment includes an execution unit 11, a storage unit 12, and a determination unit 13, for example, as described above with reference to FIGS. 2 to 4. Operate. That is, when the storage unit 12 divides the image into a plurality of pixel blocks and performs image processing on each pixel block to perform decoding processing on the image, the storage unit 12 stores the execution state of the image processing for each pixel block. .. The determination unit 13 refers to the first pixel block based on the execution state of the image processing for the second pixel block in which the image processing for the first pixel block has a dependency relationship with respect to the processing order, which is stored in the storage unit 12. Determine if image processing is feasible. The execution unit 11 executes image processing for a plurality of first pixel blocks determined by the determination unit 13 in parallel, and an image for the first pixel block stored in the storage unit 12. Update the execution status of the process.

FIG. 5 is a diagram showing a comparative example of the order in which the moving image decoding device 10 according to the present embodiment and the general moving image decoding device execute prediction processing (image processing) for pixel blocks in parallel. In FIG. 5, the white square represents the inter-prediction block and the gray square represents the intra-prediction block. In FIG. 5, the numbers assigned to the intra prediction blocks represent the processing order for each intra prediction block, and the intra prediction blocks to which the same numbers are assigned can be processed in parallel. Further, the reference adjacent block for each intra prediction block is a pixel block located on the left, upper left, upper, and upper right of the intra prediction block.

In the example shown in FIG. 5, it is assumed that the reference adjacent blocks for each intra prediction block are all inter prediction blocks. The moving image decoding device 10 according to the present embodiment manages the execution state of the prediction process for the reference adjacent block (second pixel block) for each pixel block (first pixel block). Therefore, as shown in FIG. 5A, the moving image decoding device 10 can execute the prediction processing for the six intra prediction blocks in parallel. That is, in this case, the maximum degree of parallelism of processing is "6", and the number of processing steps is "1".

On the other hand, a general moving image decoding device searches for the intra prediction block to be processed in order from the upper left of the frame. Therefore, a general moving image decoding device can start the prediction process for the intra prediction block (m, n) in the range of the coordinate values (1, n) to the coordinate values (m-1, n) (that is, the intra). This is after the prediction process for the intra prediction block located on the left side of the column where the y coordinate is the same as the prediction block (m, n) is completed. Therefore, as shown in FIG. 5B, a general moving image decoding device executes prediction processing for six intra prediction blocks in order in parallel. That is, in this case, the maximum degree of parallelism of processing is "3", and the number of processing steps is "3".

As described above, the moving image decoding device 10 according to the present embodiment reduces the number of steps of the prediction processing by increasing the degree of parallelism of the prediction processing with respect to the pixel block as compared with the general moving image decoding device. , The decoding process for moving images can be speeded up. In recent years, devices such as GPUs (Graphics Processing Units) that can execute a large number of processes in parallel by providing a large number of arithmetic units have been realized. The moving image decoding device 10 according to the present embodiment can speed up the decoding process for moving images by effectively utilizing the functions provided by such a device.

<Second embodiment>
The configuration of the moving image decoding device 10A according to the second embodiment of the present invention will be described with reference to FIG. In the present embodiment, the same number as that of the first embodiment is assigned to the configuration having the same function as that of the first embodiment described above, and detailed description thereof will be omitted.

The moving image decoding device 10A according to the first embodiment refers to the intra prediction mode set for each intra prediction block when performing the prediction processing for the intra prediction block. Different from device 10. The reference adjacent blocks according to the first embodiment are fixed to pixel blocks located on, for example, left, upper left, upper, and upper right of the intra prediction block. That is, in the first embodiment, the positional relationship between the intra prediction block and the reference adjacent block thereof is common to the intra prediction blocks. On the other hand, in the present embodiment, the positional relationship between the intra prediction block and the reference adjacent block thereof may be different for each intra prediction block. The intra prediction mode is information that can identify a reference adjacent block related to a certain intra prediction block.

FIG. 6 is a diagram illustrating the contents of the intra prediction mode referred to by the moving image decoding apparatus 10A according to the present embodiment. The content shown in FIG. 6 is described in Non-Patent Document 1. In the example shown in FIG. 6, for example, mode 0 indicates that the pixel block located above the intra prediction block is set as the reference adjacent block. Similarly, for example, mode 1 indicates that the pixel block located on the left side of the intra prediction block is set as the reference adjacent block. That is, in the prediction process, the intra prediction block in which the intra prediction mode is set to "0" has a dependency relationship with the pixel block located on the upper side, and the intra prediction block in which the intra prediction mode is set to "1" has a dependency relationship. , Has a dependency on the pixel block located on the left side.

As shown in FIG. 1, the moving image decoding device 10A according to the present embodiment includes an execution unit 11, a storage unit 12A, and a determination unit 13A.

The storage unit 12A stores a value indicating the intra prediction mode for each pixel block.

The determination unit 13A sequentially uses pixel blocks stored in the storage unit 12A at positions specified by the intra prediction mode as reference adjacent blocks with respect to the intra prediction block selected by the intra prediction unit 115 as the processing target. Set. Then, the determination unit 13A confirms the execution state of the prediction process for the reference adjacent blocks by referring to the storage unit 12. When all the prediction processes for the reference adjacent blocks are completed, the determination unit 13A determines that the intra prediction process can be executed by the intra prediction unit 115, and notifies the intra prediction unit 115 of the determination result.

The operation of the moving image decoding device 10A according to the present embodiment is as shown in the flowchart shown in FIG. 2, and the details of step S104 in the flowchart shown in FIG. 2 are different from those of the moving image decoding device 10 according to the first embodiment. ..

FIG. 7 is a flowchart showing the details of the operation of the moving image decoding apparatus 10A according to the present embodiment, that is, the processing of step S104, that is, the prediction processing using the intra prediction method for the pixel block.

The intra prediction unit 115 in the execution unit 11 performs the same processing as in step S301 shown in FIG. 4 (step S401). The determination unit 13A sequentially sets pixel blocks located at positions specified by the intra prediction mode with respect to the intra prediction block selected by the intra prediction unit 115 as reference adjacent blocks, and predicts the reference adjacent blocks. Check the execution status of (step S402).

If the prediction process for the reference adjacent block is not completed (No in step S403), the process returns to step S401. When the prediction processing for the reference adjacent block is completed (Yes in step S403), the determination unit 13A performs the loop processing from step S402 for the next reference adjacent block, and the processing for all the reference adjacent blocks is completed. If so, the process proceeds to step S405 (step S404). The moving image decoding device 10A performs the same processing as in steps S305 to S309 shown in FIG. 4 (step S405), and the processing in step S104 ends.

The moving image decoding device 10A according to the present embodiment can speed up the decoding process for moving images. The reason is as described with respect to the first embodiment.

Further, the moving image decoding device 10A according to the present embodiment narrows down the target for confirming whether or not the dependency has been resolved in the prediction process for the intra prediction block to the reference adjacent block specified by the intra prediction mode. The decoding process for moving images can be further speeded up.

FIG. 8 is a diagram showing a comparative example of the order in which the moving image decoding device 10A according to the present embodiment and the moving image decoding device 10 according to the first embodiment execute the prediction processing for the pixel block in parallel. .. In FIG. 8, the white and gray quadrangles and the numbers assigned to the pixel blocks are as described in FIG.

FIG. 8A illustrates an intra prediction mode for each pixel block (intra prediction block). In FIG. 8A, for example, for the pixel blocks (2, 2), the mode 2 illustrated in FIG. 6 is set as the intra prediction mode. In FIG. 8A, for example, for the pixel blocks (5, 1), the mode 1 illustrated in FIG. 6 is set as the intra prediction mode.

When the intra prediction mode for each pixel block is set as illustrated in FIG. 8A, the execution order of the prediction processing for the intra prediction block performed by the moving image decoding apparatus 10A according to the present embodiment is shown in FIG. It is as illustrated in (b). That is, in FIG. 8B, the intra-prediction block whose execution order is “1” is an inter-prediction block as illustrated in FIG. 8A, and therefore the reference adjacent blocks are all inter-prediction blocks. Can perform prediction processing for these intra prediction blocks in parallel in the first step. In FIG. 8B, the intra-prediction block whose execution order is “2” is an intra-prediction block in which the reference adjacent block is predicted in the first step as illustrated in FIG. 8 (a). That is, in this case, the maximum degree of parallelism of processing is "6", and the number of processing steps is "2".

On the other hand, when the intra prediction mode for each pixel block is set as illustrated in FIG. 8A, the execution order of the prediction processing for the intra prediction block performed by the moving image decoding apparatus 10 according to the first embodiment. Is as illustrated in FIG. 8 (c). Since the moving image decoding device 10 does not refer to the intra prediction mode set for each intra prediction block, the prediction processing for a certain intra prediction block is performed by the reference adjacent blocks (that is, the pixels located at the left, upper left, upper, and upper right). Execute after the prediction process for block) is completed. Therefore, in this case, the maximum degree of parallelism of the process is "2", and the number of steps of the process is "6".

As described above, the moving image decoding device 10A according to the present embodiment can further speed up the decoding process for the moving image as compared with the moving image decoding device 10 according to the first embodiment.

<Third embodiment>
The configuration of the moving image decoding device 10B according to the third embodiment of the present invention will be described with reference to FIG. In the present embodiment, the same number as that of the first embodiment is assigned to the configuration having the same function as that of the first embodiment described above, and detailed description thereof will be omitted.

The moving image decoding device 10B according to the present embodiment is a value representing the dependency relationship of the processing order related to the intra prediction block obtained by analyzing the dependency relationship regarding the prediction processing for the intra prediction block in advance (hereinafter, “dependency index”). ”) Is calculated. The moving image decoding device 10 and the moving image decoding device 10 according to the first embodiment are used in that the moving image decoding device 10B controls to execute the prediction processing for the intra prediction block in parallel by using the dependency index. It is different from the moving image decoding device 10A according to the second embodiment. However, the dependency index is information representing the number of reference-adjacent blocks on which the intra-prediction block depends in the prediction process, and its value is updated when the prediction process for a certain reference-adjacent block is completed. Since the prediction process for the inter-prediction block is processed in high parallel in a short time, the dependency index according to the present embodiment may be a value calculated only for the intra-prediction block.

As shown in FIG. 1, the moving image decoding device 10B according to the present embodiment includes an execution unit 11B, a storage unit 12B, and a determination unit 13B.

The execution unit 11B is stored in the storage unit 12B before the decoding process for the moving image is started by the moving image decoding device 10B, and the information that can identify the reference adjacent block for each intra prediction block (for example, the first The dependency index for each intra prediction block is calculated based on the coding type according to the embodiment and the intra prediction mode according to the second embodiment. The execution unit 11B stores the calculated dependency index for each intra prediction block in the storage unit 12B.

Each time the execution unit 11B completes the prediction process for any of the intra prediction blocks, the execution unit 11B updates (counts down) the dependency index related to the intra prediction block having the intra prediction block as the reference adjacent block by subtracting “1”.

When the dependency index for a certain intra prediction block indicates that there is no reference adjacent block for which the prediction process has not been completed (for example, the dependency index is "0"), the determination unit 13B predicts the intra prediction block. Judges as feasible.

FIG. 9 is a diagram illustrating a dependency index regarding a pixel block (intra-prediction block) updated by the moving image decoding device 10B according to the present embodiment. In FIG. 9, the white and gray rectangles are as described for FIG.

FIG. 9A illustrates an intra prediction mode for each intra prediction block. It is assumed that the intra prediction mode for each intra prediction block according to the present embodiment is equivalent to the intra prediction mode for each intra prediction block according to the second embodiment illustrated in FIG. 8A.

When the intra prediction mode for each intra prediction block is set as illustrated in FIG. 9A, the dependency index for the intra prediction block updated by the moving image decoding device 10B according to the present embodiment is shown in FIG. 9 (a). It is as illustrated in b). However, in FIG. 9B, the number assigned to the intra-prediction block represents the dependence index for the intra-prediction block.

As illustrated in FIG. 9B, 6 of the 9 intra-prediction blocks included in the frame have their dependency index initially set to 0, and 3 of the 9 intra-prediction blocks have It is assumed that the dependency index is initially set to 1. The six intra prediction blocks whose dependency index is initially set to 0 in FIG. 9B are equal to the pixel blocks in which the execution order of the prediction processing is “1” in FIG. 8B. The three intra-prediction blocks whose dependency index is initially set to 1 in FIG. 9 (b) are equal to the pixel blocks in which the execution order of the prediction process is “2” in FIG. 8 (b).

The moving image decoding device 10B executes the prediction processing for the six intra prediction blocks whose dependency index is initially set to 0 in parallel in the first step. After completing the prediction process of the first step, the moving image decoding device 10B determines the dependency index for the three intra prediction blocks whose dependency index is initially set to 1, which depends on the intra prediction block in which the prediction process is executed. , Update from "1" to "0". After that, the moving image decoding device 10B executes the prediction processing for the three intra prediction blocks in parallel in the second step. That is, in this case, the maximum degree of parallelism of processing is "6", and the number of processing steps is "2".

The operation of the moving image decoding device 10B according to the present embodiment is as shown in the flowchart shown in FIG. 2, and the details of step S104 in the flowchart shown in FIG. 2 are different from those of the moving image decoding device 10 according to the first embodiment. ..

FIG. 10 is a flowchart showing the details of the operation of the moving image decoding apparatus 10B according to the present embodiment, that is, the processing of step S104, that is, the prediction processing using the intra prediction method for the pixel block.

The execution unit 11B sequentially selects the intra prediction blocks based on the information stored in the storage unit 12B before the moving image decoding device 10B starts the decoding process for the moving image (step S501). The execution unit 11B calculates a dependency index for the selected intra prediction block based on the information stored in the storage unit 12B, and stores the calculated dependency index in the storage unit 12B (step S502). The execution unit 11B repeats the loop processing of steps S501 to S503 until the processing for all the intra prediction blocks is completed (step S103).

The intra prediction unit 115 in the execution unit 11B performs the same processing as in step S301 shown in FIG. 4 (step S504). The determination unit 13B confirms the dependency index regarding the intra prediction block selected by the intra prediction unit 115 by referring to the storage unit 12B (step S505). If the dependency index is not "0" (No in step S506), the process returns to step S504. When the dependency index is "0" (Yes in step S506), the moving image decoding apparatus 10B performs the same processing as in steps S305 to S307 shown in FIG. 4 (step S507).

The execution unit 11B subtracts “1” from the dependency index related to the selected intra prediction block (step S508). The intra prediction unit 115 performs the same processing as in step S308 shown in FIG. 4 (step S509). If there is an unprocessed intra prediction block (Yes in step S510), the process returns to step S504. If there is no unprocessed intra prediction block (No in step S510), the processing in step S104 ends.

The moving image decoding device 10B according to the present embodiment can speed up the decoding process for moving images. The reason is as described with respect to the first embodiment.

Further, the moving image decoding device 10B according to the present embodiment determines whether or not the dependency relationship has been resolved in the prediction process for the intra prediction block, and whether or not the dependency index for the intra prediction block is, for example, "0". Since it can be performed by a simple process of confirming whether or not, the decoding process for the moving image can be speeded up.

<Fourth Embodiment>
FIG. 11 is a block diagram conceptually showing the configuration of the moving image coding device 20 according to the fourth embodiment of the present invention. The moving image coding device 20 according to the present embodiment performs coding processing based on, for example, H.264 or H.265 on the input image, and as a result of the coding processing, outputs a bit stream. This is a device that outputs to a moving image decoding device (for example, the moving image decoding device 10 according to the first embodiment) (not shown). The moving image coding device 20 according to the present embodiment is a device to which the configuration (technology) provided in the moving image decoding device 10 according to the first embodiment is applied.

The moving image coding device 20 according to the present embodiment is roughly classified into an execution unit 21, a storage unit 22, a determination unit 23, a first cost calculation unit 24, a second cost calculation unit 25, and a generation unit 26. There is.

The first cost calculation unit 24 calculates the coding cost when the inter-prediction method is used as the prediction processing (image processing) for the pixel blocks for each pixel block. However, the coding cost is an index indicating the amount of information processing required to perform coding and decoding of the image. It is assumed that the first cost calculation unit 24 has a calculation standard for calculating the coding cost when the inter-prediction method is used. At this time, the first cost calculation unit 24 also generates a motion vector (prediction vector) in the inter-prediction.

The second cost calculation unit 25 calculates the coding cost for each pixel block when the intra prediction method is used as the prediction process (image processing) for the pixel block. However, it is assumed that the second cost calculation unit 25 has a calculation standard for calculating the coding cost when the intra prediction method is used.

The generation unit 26 uses the coding type information (that is, whether the coding is performed by inter-prediction or not) for each pixel block based on the calculation result regarding the coding cost by the first cost calculation unit 24 and the second cost calculation unit 25. Information that identifies whether to do it by prediction) is generated. For example, when the inter-prediction has a lower coding cost for a certain pixel block, the generation unit 26 sets the coding type for the pixel block to inter-prediction, and when the intra-prediction has a lower coding cost, the generation unit 26 sets the coding type to the inter-prediction. , Set the encoding type for the pixel block to intra prediction. The generation unit 26 stores the generated coding type information in the storage unit 22.

The storage unit 22 is a storage device such as an electronic memory or a magnetic disk. The storage unit 22 stores the execution state of the inter-prediction processing (image processing) by the inter-prediction unit 215 and the intra-prediction processing (image processing) by the intra-prediction unit 216, which will be described later, for each pixel block. These execution states are updated by the inter-prediction unit 215 or the intra-prediction unit 216. The storage unit 22 also stores the coding type information input by the generation unit.

When the intra-prediction unit 216 selects an unprocessed pixel block (first pixel block) as the intra-prediction processing target, the determination unit 23 determines whether or not the intra-prediction processing can be executed for the pixel block. Is determined. At that time, the determination unit 23 performs the same processing as the determination unit 13 according to the first embodiment. The determination unit 23 notifies the intra prediction unit 216 of the determination result.

The execution unit 21 includes a variable length coding unit 210, an inverse conversion inverse quantization unit 211, a conversion quantization unit 212, a loop filter 213, a frame buffer 214, an inter prediction unit 215, an intra prediction unit 216, and a switch 217. I'm out.

The inter-prediction unit 215 performs inter-prediction processing on the selected inter-prediction block in the same manner as the inter-prediction unit 114 according to the first embodiment. More specifically, the inter-prediction unit 215 inter-predicts based on the encoded frame (image) stored in the frame buffer 214 described later and the motion vector generated by the first cost calculation unit 24. A predicted image showing the result of the processing is generated. The inter-prediction unit 215 updates the execution state stored in the storage unit 22 to "completed" with respect to the inter-prediction block that has completed the prediction process. The execution unit 21 generates a difference between the prediction image generated by the inter-prediction unit 215 and the input image as a residual signal.

The intra prediction unit 216 performs the intra prediction processing on the selected intra prediction block in the same manner as the intra prediction unit 115 according to the first embodiment. More specifically, the intra prediction unit 216 generates a prediction image representing the result of performing the intra prediction processing by referring to the reconstructed image (described later) relating to the adjacent reference block. The intra prediction unit 216 updates the execution state stored in the storage unit 22 to "completed" with respect to the intra prediction block that has completed the prediction process. The execution unit 21 generates a difference between the prediction image generated by the intra prediction unit 216 and the input image as a residual signal.

The execution unit 21 appropriately performs the above-mentioned inter-prediction processing and intra-prediction processing by switching the switch 217 that switches the output of the inter-prediction unit 215 and the intra-prediction unit 216 according to the coding type of the pixel block to be processed. It shall be possible to switch.

The conversion quantization unit 212 quantizes the conversion coefficient by orthogonally converting the residual signal generated by the execution unit 21. Inverse transformation The inverse quantization unit 211 performs inverse orthogonal transformation after inverse quantizing the quantized conversion coefficient. The execution unit 21 generates a reconstructed image by adding the result of the inverse orthogonal conversion to the prediction image generated by the inter prediction unit 215 or the intra prediction unit 216.

The variable length coding unit 210 outputs a bit stream generated by encoding the conversion coefficient quantized by the conversion quantization unit 212. The loop filter 213 stores the frame (image) generated by removing the block distortion of the reconstructed image generated by the execution unit 21 in the frame buffer 214.

Next, the operation (processing) of the moving image coding device 20 according to the present embodiment will be described in detail with reference to the flowcharts of FIGS. 12 to 14.

FIG. 12 is a flowchart showing the entire operation of the moving image coding device 20.

The first cost calculation unit 24 calculates the coding cost when the inter-prediction method is used (step S601). The second cost calculation unit 25 calculates the coding cost when the intra prediction method is used (step S602). The generation unit 26 generates the coding type information including the coding type determined based on the calculation result by the first cost calculation unit 24 and the second cost calculation unit 25, and stores the coding type information in the storage unit 22. (Step S603).

The execution unit 21 performs inter-prediction processing, conversion quantization processing, and inverse conversion inverse quantization processing on a pixel block (inter-prediction block) whose coding type is inter-prediction to produce a reconstructed image. Generate (step S604). The execution unit 21 performs an intra-prediction process, a conversion quantization process, and an inverse conversion inverse quantization process on a pixel block (intra-prediction block) whose coding type is intra-prediction to produce a reconstructed image. Generate (step S605).

The variable-length coding unit 210 in the execution unit 21 generates an output bit stream by encoding the conversion coefficient quantized by the conversion quantization unit 212 (step S606). The loop filter 213 in the execution unit 21 stores the result of removing the block distortion of the reconstructed image generated by the execution unit 21 in the frame buffer 214 (step S607), and the entire process ends.

FIG. 13 is a flowchart showing the details of the operation in which the moving image coding device 20 performs the prediction processing using the inter-prediction method on the pixel block. That is, FIG. 13 is a flowchart showing the details of step S604 in the flowchart shown in FIG.

The inter-prediction unit 215 in the execution unit 21 selects an unprocessed inter-prediction block by referring to the execution state of the prediction process for each pixel block stored in the storage unit 22 (step S701). The execution unit 21 generates a reconstructed image by performing inter-prediction processing, block conversion quantization processing, and block inverse conversion inverse quantization processing on the selected inter-prediction block (step S702).

The inter-prediction unit 215 in the execution unit 21 updates the execution state stored in the storage unit 22 to "completed" with respect to the inter-prediction block that has completed the prediction process (step S703). The inter-prediction unit 215 confirms whether or not there is an unprocessed inter-prediction block by referring to the storage unit 22 (step S704).

If there is an unprocessed inter-prediction block (Yes in step S705), processing returns to step S701. If there is no unprocessed inter-prediction block (No in step S705), the processing in step S604 ends.

FIG. 14 is a flowchart showing the details of the operation in which the moving image coding device 20 performs the prediction processing using the intra prediction method on the pixel block. That is, FIG. 14 is a flowchart showing the details of step S605 in the flowchart shown in FIG.

The intra prediction unit 216 in the execution unit 21 selects an unprocessed intra prediction block by referring to the execution state of the prediction process for each pixel block stored in the storage unit 22 (step S801). The determination unit 23 sequentially sets the pixel blocks located on the left, upper left, upper, and upper right of the intra prediction block selected by the intra prediction unit 216 as reference adjacent blocks, and confirms the execution state (the execution state). Step S802).

If the prediction process for the reference adjacent block is not completed (No in step S803), the process returns to step S801. When the prediction processing for the reference adjacent block is completed (Yes in step S803), the determination unit 23 performs the loop processing from step S802 for the next reference adjacent block, and the processing for all the reference adjacent blocks is completed. If so, the process proceeds to step S805 (step S804).

The execution unit 21 generates a reconstructed image by performing the intra prediction process, the block conversion quantization process, and the block inverse conversion inverse quantization process on the selected intra prediction block (step S805). The intra-prediction unit 216 updates the execution state stored in the storage unit 22 to "completed" with respect to the intra-prediction block for which the prediction process has been completed (step S806).

The intra prediction unit 216 confirms whether or not there is an unprocessed intra prediction block by referring to the storage unit 22 (step S807). If there is an unprocessed intra prediction block (Yes in step S808), the process returns to step S801. If there is no unprocessed inter-prediction block (No in step S808), the processing in step S605 ends.

The moving image coding device 20 according to the present embodiment can speed up the coding process for the moving image. The reason is as described with respect to the first embodiment.

Further, as the moving image coding device 20 according to the present embodiment, the configuration (technology) provided in the moving image decoding device 10A according to the second embodiment or the moving image decoding device 10B according to the third embodiment is applied. It may be a device that has been installed.

<Fifth Embodiment>
FIG. 15 is a block diagram conceptually showing the configuration of the moving image processing device 30 according to the fifth embodiment of the present invention.

The moving image processing device 30 according to the present embodiment includes an execution unit 31, a storage unit 32, and a determination unit 33.

When the storage unit 32 divides an image into a plurality of pixel blocks and performs image processing on each pixel block to perform coding processing or decoding processing on the image, the storage unit 32 is in an execution state of image processing for each pixel block. Remember.

The determination unit 33 refers to the first pixel block based on the execution state of the image processing for the second pixel block in which the image processing for the first pixel block has a dependency relationship with respect to the processing order, which is stored in the storage unit 32. Determine if image processing is feasible.

The execution unit 31 executes image processing on a plurality of first pixel blocks determined by the determination unit 33 to be executable in parallel or pseudo-parallel. The execution unit 31 also updates the execution state of image processing for the first pixel block stored in the storage unit 32.

The moving image processing device 30 according to the present embodiment can speed up image processing for moving images. The reason is that the moving image processing device 30 manages the execution state of image processing for each pixel block when the first pixel block has a dependency relationship with respect to the second pixel block regarding the image processing. This is because it is possible to detect that the dependency has been resolved at an early timing, and as a result, accelerate the start of image processing for the first pixel block.

<Hardware configuration example>
In each of the above-described embodiments, each part of the moving image decoding device 10 (10A, 10B) shown in FIG. 1, the moving image coding device 20 shown in FIG. 11, and the moving image processing device 30 shown in FIG. 15 is , Can be realized by a dedicated HW (HardWare) (electronic circuit). Further, in FIGS. 1, 11, and 15, at least the following configuration can be regarded as a function (processing) unit (software module) of the software program.
-Executive units 11 (11B), 21, and 31,
Memory control functions in the storage units 12 (12A, 12B), 22, and 32,
Judgment units 13 (13A, 13B), 23, and 33,
・ First cost calculation unit 24,
・ Second cost calculation unit 25,
-Generation unit 26.

However, the division of each part shown in these drawings is a configuration for convenience of explanation, and various configurations can be assumed at the time of mounting. An example of the hardware environment in this case will be described with reference to FIG.

FIG. 16 shows a configuration of an information processing device 900 (computer) capable of executing a moving image decoding device 10 (10A, 10B), a moving image coding device 20, and a moving image processing device 30 according to each embodiment of the present invention. It is a figure for exemplifying. That is, FIG. 16 shows the moving image decoding device 10 (10A, 10B), the moving image coding device 20, the moving image processing device 30, or a part thereof shown in FIGS. 1, 11 and 15. It is a feasible computer (information processing device) configuration, and represents a hardware environment in which each function in the above-described embodiment can be realized. The information processing apparatus 900 shown in FIG. 16 includes the following as components.
-CPU (Central_Processing_Unit) 901,
-ROM (Read_Only_Memory) 902,
・ RAM (Random_Access_Memory) 903,
-Hard disk (storage device) 904,
-Communication interface 905 with an external device,
・ Bus 906 (communication line),
A reader / writer 908 that can read and write data stored in a recording medium 907 such as a CD-ROM (Compact_Disc_Read_Only_Memory),
-Input / output interface 909.

That is, the information processing device 900 including the above components is a general computer in which these components are connected via the bus 906. The information processing device 900 may include a plurality of CPUs 901, or may include a CPU 901 configured by a multi-core processor. The information processing apparatus 900 may also have a configuration in which a main general-purpose CPU and a hardware accelerator specialized for specific arithmetic processing cooperate with each other so that a plurality of processes can be executed in parallel.

Then, the present invention described by taking the above-described embodiment as an example supplies the computer program capable of realizing the following functions to the information processing apparatus 900 shown in FIG. The function is the above-described configuration in the block configuration diagram (FIGS. 1, 11, and 15) referred to in the description of the embodiment, or the flowchart (FIGS. 2 to 4, FIG. 7, FIG. 10, and FIG. 12). It is a function of to 14). The present invention is then achieved by reading, interpreting, and executing the computer program in the CPU 901 of the hardware. Further, the computer program supplied in the device may be stored in a readable / writable volatile memory (RAM 903) or a non-volatile storage device such as a ROM 902 or a hard disk 904.

Further, in the above case, as a method of supplying the computer program into the hardware, a general procedure can be adopted at present. As the procedure, for example, there are a method of installing in the device via various recording media 907 such as a CD-ROM, a method of downloading from the outside via a communication line such as the Internet, and the like. In such a case, the present invention can be regarded as being composed of a code constituting the computer program or a recording medium 907 in which the code is stored.

The invention of the present application has been described above using the above-described embodiment as a model example. However, the present invention is not limited to the above-described embodiments. That is, the present invention can apply various aspects that can be understood by those skilled in the art within the scope of the present invention.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2017-077532 filed on April 10, 2017, the entire disclosure of which is incorporated herein by reference.

10 Video decoding device 11 Execution unit 110 Variable length decoding unit 111 Inverse conversion inverse quantization unit 112 Loop filter 113 Frame buffer 114 Inter prediction unit 115 Intra prediction unit 116 and 117 Switch 12 Storage unit 13 Judgment unit 20 Video encoding device 21 Execution unit 210 Variable length coding unit 211 Inverse conversion inverse quantization unit 212 Conversion quantization unit 213 Loop filter 214 Frame buffer 215 Inter prediction unit 216 Intra prediction unit 217 Switch 22 Storage unit 23 Judgment unit 24 First cost calculation unit 25 2nd cost calculation unit 30 Video processing device 31 Execution unit 32 Storage unit 33 Judgment unit 40 Video decoding device 410 Variable length decoding unit 411 Inverse conversion inverse quantization unit 412 Loop filter 413 Frame buffer 414 Inter prediction unit 415 Intra prediction unit 900 Information processing device 901 CPU
902 ROM
903 RAM
904 hard disk (storage device)
905 Communication interface 906 Bus 907 Recording medium 908 Reader / writer 909 Input / output interface

Claims (7)

When an image is divided into a plurality of pixel blocks and image processing is performed on each of the pixel blocks to perform coding processing or decoding processing on the image, the execution state of the image processing is stored for each pixel block. Memories to do and
Image processing for the first pixel block based on the execution state of image processing for the second pixel block in which the image processing for the first pixel block stored in the storage means has a dependency relationship with respect to the processing order. As a means of determining whether or not is feasible,
The determination means executes image processing for a plurality of the first pixel blocks determined to be feasible in parallel or pseudo-parallel, and for the first pixel block stored in the storage means. Execution means to update the execution status of image processing and
With
The executing means generates a predicted image regarding the first pixel block by using information representing the second pixel block adjacent to the first pixel block.
The storage means includes an inter-prediction method that generates the predicted image based on temporal correlation and an intra-prediction method that generates the predicted image based on spatial correlation as image processing for the pixel block. Encoding type information indicating which of the above is used is stored for each pixel block, and when the decoding process is performed in the intra prediction method, the second pixel block adjacent to the first pixel block is stored. A value indicating an intra-prediction mode for specifying a pixel block of the above is stored for each pixel block.
The determination unit is In no event where the encoding type information indicating that the use of the intra-prediction method, when performing the decryption process, the second pixels specified by the value indicating the intra prediction mode The image processing for the first pixel block is executed when the second pixel block indicating that the execution state of the image processing is incomplete does not exist by referring to the execution state of the image processing for the block. Judge as possible,
Moving image processing device. A first cost calculation means for calculating the coding cost when the inter-prediction method is used as the image processing for the pixel block in the coding process.
A second cost calculation means for calculating the coding cost when the intra prediction method is used as the image processing for the pixel block in the coding process.
Based on the calculation results by the first and second cost calculation means, the coding type information is generated for each pixel block, and the generated coding type information is stored in the storage means.
The moving image processing apparatus according to claim 1. Before the coding process or the decoding process for the image is started, the storage means sets a value representing the dependency relationship between the first pixel block and the second pixel block using the intra prediction method. , Stored for each pixel block,
Each time the executing means completes image processing for any of the second pixel blocks, the executing means updates a value representing the dependency with respect to the first pixel block.
The determination means refers to the first pixel block when the value representing the dependency relationship with respect to the first pixel block indicates that the second pixel block for which the image processing has not been completed does not exist. Judge that image processing is feasible,
The moving image processing apparatus according to claim 1 or 2. The executing means calculates and calculates a value representing the dependency relationship with respect to the pixel block in which the image processing using the intra prediction method is performed before the coding process or the decoding process for the image is started. A value representing the dependency is stored in the storage means.
The moving image processing apparatus according to claim 3. The execution means is H.I. 264 and H. Perform the image processing according to any of the standards for encoding moving images, including 265.
The moving image processing apparatus according to any one of claims 1 to 4. Depending on the information processing device
When an image is divided into a plurality of pixel blocks and image processing is performed on each of the pixel blocks to perform coding processing or decoding processing on the image, the execution state of the image processing is stored for each pixel block. Remember in the means,
Image processing for the first pixel block based on the execution state of image processing for the second pixel block in which the image processing for the first pixel block stored in the storage means has a dependency relationship with respect to the processing order. Determines if is feasible and
The image processing for the plurality of first pixel blocks determined to be feasible is executed in parallel or pseudo-parallel, and the execution state of the image processing for the first pixel block stored in the storage means is set. How to update
Using the information representing the second pixel block adjacent to the first pixel block, a predicted image relating to the first pixel block is generated.
As image processing for the pixel block, either an inter-prediction method that generates the predicted image based on temporal correlation or an intra-prediction method that generates the predicted image based on spatial correlation is used. When the coding type information indicating whether or not to be performed is stored in the storage means for each pixel block and the decoding process is performed in the intra prediction method, the second pixel block adjacent to the first pixel block is stored. A value indicating an intra prediction mode for identifying a pixel block is stored in the storage means for each pixel block.
In no event indicating that the encoding type information using said intra prediction method, when performing the decoding processing, the image processing for the second pixel block identified by the value indicating the intra prediction mode When the second pixel block indicating that the execution state of the image processing is incomplete disappears by referring to the execution state, it is determined that the image processing for the first pixel block can be executed.
Moving image processing method. When an image is divided into a plurality of pixel blocks and image processing is performed on each of the pixel blocks to perform coding processing or decoding processing on the image, the execution state of the image processing is stored for each pixel block. Memory control processing to be stored in the means and
Image processing for the first pixel block based on the execution state of image processing for the second pixel block in which the image processing for the first pixel block stored in the storage means has a dependency relationship with respect to the processing order. Judgment processing to determine whether or not is feasible,
By the determination process, the image processing for the plurality of the first pixel blocks determined to be executable is executed in parallel or pseudo-parallel, and the first pixel block stored in the storage means is subjected to the image processing. Execution processing that updates the execution status of image processing and
Is a program that allows a computer to execute
The execution process generates a predicted image of the first pixel block by using the information representing the second pixel block adjacent to the first pixel block.
The memory control process is an inter-prediction method that generates the predicted image based on temporal correlation and an intra-prediction method that generates the predicted image based on spatial correlation as image processing for the pixel block. Coding type information indicating which of the above is used is stored in the storage means for each pixel block, and when the decoding process is performed in the intra prediction method, the first pixel block is stored. A value indicating an intra prediction mode that identifies an adjacent second pixel block is stored in the storage means for each pixel block.
The determination processing In no event where the encoding type information indicating that the use of the intra-prediction method, when performing the decryption process, the second pixels specified by the value indicating the intra prediction mode The image processing for the first pixel block is executed when the second pixel block indicating that the execution state of the image processing is incomplete does not exist by referring to the execution state of the image processing for the block. Judge as possible,
Video processing program.

Images