JP2016106483A - Image processing device and method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress decrease in coding efficiency due to high speed of image coding.SOLUTION: A slice 1 processing section 153 encodes each macro block line of slice 1 from the bottom to the top. The slice 1 processing section 153 encodes each macro block depending on the state of a macro block just left of the processed macro block, and a macro block just below the processed macro block. A slice 2 processing section 154 encodes each macro block line of slice 2 from top to bottom. The slice 2 processing section 154 encodes each macro block depending on the state of a macro block just left of the processed macro block, and a macro block just above the processed macro block. The invention is applicable to an image processing device, for example.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing apparatus and method, a program, and a recording medium, and in particular, an image processing apparatus and method, a program, and a program capable of suppressing reduction in encoding efficiency due to high-speed image encoding, and The invention relates to a recording medium.

  Conventionally, in an AVC (Advanced Video Coding) image coding method, CABAC (Context-based Adaptive Binary Arithmetic Coding) and CAVLC (Context-based Adaptive Variable Length Coding) are defined as entropy coding. Among them, CABAC is a binary arithmetic coding system that performs coding adaptively according to the surrounding situation (context).

  In arithmetic coding, a code string is obtained by repeating the process of dividing a numerical interval according to the occurrence probability for each symbol. That is, since it is necessary to process all symbols sequentially, it is difficult to parallelize the processing, and thus it is difficult to increase the processing speed.

  Therefore, AVC has made it possible to achieve high throughput using a technique called multi slice. In multi-slice, as shown in FIG. 1A, an image is divided into a plurality of regions, and CABAC encoding processing is performed independently on each of the regions. That is, as indicated by the dotted arrow in FIG. 1A, arithmetic coding is performed for each macroblock in the order from the upper macroblock line in the image to the lower macroblock line in the image. In this way, CABAC encoding of each region can be performed in parallel, and the processing speed is increased. This divided area is referred to as a slice.

  Each macroblock is encoded with reference to other adjacent macroblocks as shown in FIGS. 1B and 1C.

  However, dividing the area becomes a factor of reducing the coding efficiency. First, since each slice is encoded independently, as shown in FIG. 1B and FIG. 1C, it is not possible to cross-reference in encoding each region (slice). Therefore, intra-prediction cannot be used at the boundary portion, which may reduce the coding efficiency.

  Second, in CABAC, in order to select the symbol occurrence probability table, the state of neighboring macroblocks is referred to as shown in FIG. 1C. It may be difficult to select an appropriate occurrence probability table.

  Third, the symbol occurrence probability table used in CABAC converges to the actual occurrence probability as encoding progresses, and the encoding efficiency increases. However, in the case of multi slice, for each region (slice) Since the symbol occurrence probability table is initialized, the coding efficiency at the start of the slice may be reduced.

  An entropy slice method has been proposed as a method for improving coding efficiency while parallelizing CABAC (see, for example, Non-Patent Document 1). According to this method, only the CABAC processing part is divided into regions called entropy slices and executed in parallel, so that intra-screen prediction can be used even at entropy slice boundaries. It was. However, surrounding macroblock references in CABAC processing cannot be used at entropy slice boundaries.

  As a further improvement method, an ordered entropy slices method has been proposed (for example, see Patent Document 2). This method makes it possible to refer to adjacent macroblocks at the boundary even in CABAC processing.

A. Segall, J. Zhao, “Entropy slices for parallel entropy decoding”, VCEG input document COM16-C405, Geneva, CH, April 2008 Xun Guo, Yu-Wen Huang, Shawmin Lei "Ordered Entropy Slices for Parallel CABAC", VCEG input document VCEG-AK25, Yokohama, Japan, April 2009

  However, when coding the second entropy slice, the macroblock of the first entropy slice adjacent on the upper side must finish the processing, so the dependency between the slices. The benefits of parallelization were lost. In other words, it has been difficult to achieve high speed by CABAC processing parallelization, which is the original purpose.

  The present invention has been made in view of such a situation, and an object thereof is to suppress a reduction in encoding efficiency due to high-speed image encoding.

  One aspect of the present technology is that an encoding unit that encodes a block of image data using a context and the target block line in an upper block line that is a block line adjacent to an upper part of the target block line to be encoded When encoding the target head block using the context updated when the upper block, which is a block located at a different location from the upper head block adjacent to the top of the target head block located at the top of the target block, is encoded. The image processing apparatus includes a control unit that controls the encoding unit so as to initialize a context to be used.

  The control unit sets, as an initial value, a context updated when the upper block located at a position later in the encoding order than the upper head block is encoded with respect to the upper block line, the target head block The encoding unit can be controlled to encode.

  The control unit may control the encoding unit so as to encode the target head block using an initial value as a probability table updated when the upper block is encoded.

  The encoding unit may arithmetically encode the target block line using a context.

  According to another aspect of the present technology, a block of image data is encoded using a context, and an upper block line that is a block line adjacent to an upper portion of the target block line to be encoded is placed at the head of the target block line. Context used when encoding the target head block using the context updated when the upper block, which is a block located at a different location from the upper head block adjacent to the upper part of the target head block located, is encoded Is an image processing method for controlling to initialize the image.

  The target head block is encoded with the context updated when the upper block located in the encoding order after the upper head block is encoded with respect to the upper block line as an initial value. Can be controlled.

  It is possible to control to encode the target head block by using, as an initial value, the probability table updated when the upper block is encoded.

  The target block line may be arithmetically encoded using a context.

  One aspect of the present technology further includes an encoding unit that encodes a block of image data using a context, and an upper block line that is a block line adjacent to an upper part of a target block line to be encoded. The target head block using the context updated when the upper block, which is a block located at a different location from the upper head block adjacent to the top of the target head block located at the head of the target block line, is encoded. Is a program for functioning as a control unit for controlling the encoding unit so as to initialize a context used when encoding the.

  One aspect of the present technology is also that an encoding unit that encodes a block of image data using a context in a computer, and an upper block line that is a block line adjacent to an upper part of a target block line to be encoded The target head block using the context updated when the upper block, which is a block located at a different location from the upper head block adjacent to the top of the target head block located at the head of the target block line, is encoded. This is a computer-readable recording medium that records a program for functioning as a control unit for controlling the encoding unit so as to initialize a context used when encoding the video.

  In one aspect of the present technology, a block of image data is encoded using a context, and the upper block line, which is a block line adjacent to the upper part of the target block line to be encoded, is positioned at the head of the target block line. The context used when encoding the target head block is initialized using the context updated when the upper block, which is a block located at a different location from the upper head block adjacent to the upper part of the target head block to be encoded, is used. It is controlled to become.

  According to the present invention, image data can be encoded. In particular, it is possible to suppress a reduction in encoding efficiency due to high-speed image encoding.

It is a figure explaining the example of the conventional process advancing direction and a reference direction. It is a block diagram which shows the main structural examples of the image coding apparatus to which this invention is applied. It is a block diagram which shows the main structural examples of a lossless encoding part. It is a figure explaining the example of the process advancing direction and reference direction of this invention. It is a flowchart explaining the example of the flow of an encoding process. It is a flowchart explaining the example of the flow of a lossless encoding process. It is a flowchart explaining the example of the flow of a slice 1 encoding process. It is a flowchart explaining the example of the flow of a slice 2 encoding process. It is a block diagram which shows the main structural examples of the image decoding apparatus to which this invention is applied. It is a block diagram which shows the main structural examples of a lossless decoding part. It is a flowchart explaining the example of the flow of a decoding process. It is a flowchart explaining the example of the flow of a lossless decoding process. It is a flowchart explaining the example of the flow of a slice 1 decoding process. It is a flowchart explaining the example of the flow of a slice 2 decoding process. It is a block diagram which shows the other structural example of a lossless encoding part. It is a flowchart explaining the example of the flow of a lossless encoding process. It is a flowchart explaining the other example of the flow of a slice 1 encoding process. It is a flowchart explaining the other example of the flow of a slice 2 encoding process. It is a figure explaining the example of the mode of utilization of a probability table. It is a flowchart explaining the other example of the flow of a slice 1 encoding process. It is a flowchart explaining the other example of the flow of a slice 2 encoding process. It is a flowchart explaining the example of 4 division. It is a block diagram which shows the other structural example of a lossless encoding part. It is a block diagram which shows the other structural example of a lossless decoding part. It is a figure which shows the other example of a macroblock. It is a block diagram which shows the main structural examples of the personal computer to which this invention is applied. It is a block diagram which shows the main structural examples of the television receiver to which this invention is applied. It is a block diagram which shows the main structural examples of the mobile telephone to which this invention is applied. It is a block diagram which shows the main structural examples of the hard disk recorder to which this invention is applied. It is a block diagram which shows the main structural examples of the camera to which this invention is applied.

Hereinafter, modes for carrying out the invention (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First Embodiment (Image Encoding Device)
2. Second embodiment (image decoding apparatus)
3. Third Embodiment (Image Encoding Device / Image Decoding Device)
4). Fourth embodiment (personal computer)
5. Fifth embodiment (television receiver)
6). Sixth embodiment (mobile phone)
7). Seventh embodiment (hard disk recorder)
8). Eighth embodiment (camera)

<1. First Embodiment>
[Image encoding device]
FIG. 2 shows a configuration of an embodiment of an image encoding apparatus as an image processing apparatus to which the present invention is applied.

  The image encoding device 100 shown in FIG. This is an encoding device that compresses and encodes an image using H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) (hereinafter referred to as H.264 / AVC). However, the image encoding apparatus 100 performs CABAC by a multi-slice method as a lossless encoding method.

  In the example of FIG. 2, the image encoding device 100 includes an A / D (Analog / Digital) conversion unit 101, a screen rearrangement buffer 102, a calculation unit 103, an orthogonal conversion unit 104, a quantization unit 105, and a lossless encoding unit 106. And a storage buffer 107. In addition, the image coding apparatus 100 includes an inverse quantization unit 108, an inverse orthogonal transform unit 109, and a calculation unit 110. Further, the image encoding device 100 includes a deblock filter 111 and a frame memory 112. In addition, the image encoding device 100 includes a selection unit 113, an intra prediction unit 114, a motion prediction compensation unit 115, and a selection unit 116. Furthermore, the image encoding device 100 includes a rate control unit 117.

  The A / D conversion unit 101 performs A / D conversion on the input image data, and outputs to the screen rearrangement buffer 102 for storage. The screen rearrangement buffer 102 rearranges the stored frame images in the display order in the order of frames for encoding in accordance with the GOP (Group of Picture) structure. The screen rearrangement buffer 102 supplies the image with the rearranged frame order to the arithmetic unit 103, the intra prediction unit 114, and the motion prediction compensation unit 115.

  The calculation unit 103 subtracts the predicted image supplied from the selection unit 116 from the image read from the screen rearrangement buffer 102 and outputs the difference information to the orthogonal transform unit 104. For example, in the case of an image on which intra coding is performed, the calculation unit 103 adds the predicted image supplied from the intra prediction unit 114 to the image read from the screen rearrangement buffer 102. For example, in the case of an image on which inter coding is performed, the calculation unit 103 adds the predicted image supplied from the motion prediction / compensation unit 115 to the image read from the screen rearrangement buffer 102.

  The orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information from the operation unit 103 and supplies the transform coefficient to the quantization unit 105. The quantization unit 105 quantizes the transform coefficient output from the orthogonal transform unit 104. The quantization unit 105 supplies the quantized transform coefficient to the lossless encoding unit 106.

  The lossless encoding unit 106 performs CABAC on the quantized transform coefficient by the multi-slice method. That is, the lossless encoding unit 106 divides the quantized image region of the transform coefficient into two upper and lower entropy slices (slice 1 and slice 2), and performs CABAC for each slice in parallel with each other.

  At this time, the lossless encoding unit 106 advances CABAC for the upper entropy slice (slice 1) in order from the lower macroblock to the upper macroblock. Further, the lossless encoding unit 106 advances CABAC for the lower entropy slice (slice 2) in order from the upper macroblock to the lower macroblock.

  In this way, the lossless encoding unit 106 sequentially processes each macroblock in the direction away from each other's boundary with respect to the upper and lower two entropy slices.

  The lossless encoding unit 106 acquires information indicating intra prediction from the intra prediction unit 114 and acquires information indicating inter prediction mode from the motion prediction compensation unit 115. Note that information indicating intra prediction is hereinafter also referred to as intra prediction mode information. In addition, information indicating an information mode indicating inter prediction is hereinafter also referred to as inter prediction mode information.

  The lossless encoding unit 106 encodes the quantized transform coefficient, and uses a filter coefficient, intra prediction mode information, inter prediction mode information, a quantization parameter, and the like as part of the header information of the encoded data. (Multiplex). The lossless encoding unit 106 supplies the encoded data obtained by encoding to the accumulation buffer 107 for accumulation.

  The accumulation buffer 107 temporarily holds the encoded data supplied from the lossless encoding unit 106, and at a predetermined timing, the H.264 buffer stores the encoded data. As an encoded image encoded by the H.264 / AVC format, for example, it is output to a recording device or a transmission path (not shown) in the subsequent stage.

  The transform coefficient quantized by the quantization unit 105 is also supplied to the inverse quantization unit 108. The inverse quantization unit 108 inversely quantizes the quantized transform coefficient by a method corresponding to the quantization by the quantization unit 105, and supplies the obtained transform coefficient to the inverse orthogonal transform unit 109.

  The inverse orthogonal transform unit 109 performs inverse orthogonal transform on the supplied transform coefficient by a method corresponding to the orthogonal transform process by the orthogonal transform unit 104. The output subjected to inverse orthogonal transform is supplied to the calculation unit 110.

  The calculation unit 110 adds the prediction image supplied from the selection unit 116 to the inverse orthogonal transformation result supplied from the inverse orthogonal transformation unit 109, that is, the restored difference information, and generates a locally decoded image (decoding Image). For example, when the difference information corresponds to an image on which intra coding is performed, the calculation unit 110 adds the predicted image supplied from the intra prediction unit 114 to the difference information. For example, when the difference information corresponds to an image on which inter coding is performed, the arithmetic unit 110 adds the predicted image supplied from the motion prediction / compensation unit 115 to the difference information.

  The addition result is supplied to the deblock filter 111 or the frame memory 112.

  The deblocking filter 111 removes block distortion of the decoded image by appropriately performing the deblocking filter process, and improves the image quality by appropriately performing the loop filter process using, for example, a Wiener filter. The deblocking filter 111 classifies each pixel and performs an appropriate filter process for each class. The deblocking filter 111 supplies the filter processing result to the frame memory 112.

  The frame memory 112 outputs the accumulated reference image to the intra prediction unit 114 or the motion prediction compensation unit 115 via the selection unit 113 at a predetermined timing.

  For example, in the case of an image on which intra coding is performed, the frame memory 112 supplies the reference image to the intra prediction unit 114 via the selection unit 113. For example, in the case of an image on which inter coding is performed, the frame memory 112 supplies the reference image to the motion prediction / compensation unit 115 via the selection unit 113.

  In the image encoding device 100, for example, an I picture, a B picture, and a P picture from the screen rearrangement buffer 102 are supplied to the intra prediction unit 114 as images for intra prediction (also referred to as intra processing). In addition, the B picture and the P picture read from the screen rearrangement buffer 102 are supplied to the motion prediction / compensation unit 115 as an image to be inter predicted (also referred to as inter processing).

  The selection unit 113 supplies the reference image supplied from the frame memory 112 to the intra prediction unit 114 in the case of an image to be subjected to intra coding, and to the motion prediction compensation unit 115 in the case of an image to be subjected to inter coding. .

  The intra prediction unit 114 performs intra prediction (intra-screen prediction) that generates a predicted image using pixel values in the screen. The intra prediction unit 114 performs intra prediction in a plurality of modes (intra prediction modes). The intra prediction mode includes a mode for generating a prediction image based on a reference image supplied from the frame memory 112 via the selection unit 113.

  The intra prediction unit 114 generates prediction images in all intra prediction modes, evaluates each prediction image, and selects an optimal mode. When the optimal intra prediction mode is selected, the intra prediction unit 114 supplies the prediction image generated in the optimal mode to the calculation unit 103 via the selection unit 116. Further, as described above, the intra prediction unit 114 supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoding unit 106 as appropriate.

  The motion prediction / compensation unit 115 obtains an input image supplied from the screen rearrangement buffer 102 and a decoded image serving as a reference frame supplied from the frame memory 112 via the selection unit 113 for an image to be inter-coded. To calculate a motion vector. The motion prediction / compensation unit 115 performs motion compensation processing according to the calculated motion vector, and generates a prediction image (inter prediction image information).

  The motion prediction / compensation unit 115 performs inter prediction processing in all candidate inter prediction modes, and generates a prediction image. The motion prediction / compensation unit 115 supplies the generated prediction image to the calculation unit 103 via the selection unit 116.

  The motion prediction / compensation unit 115 supplies the inter prediction mode information indicating the adopted inter prediction mode and the motion vector information indicating the calculated motion vector to the lossless encoding unit 106.

  The selection unit 116 supplies the output of the intra prediction unit 114 to the calculation unit 103 in the case of an image to be subjected to intra coding, and supplies the output of the motion prediction compensation unit 115 to the calculation unit 103 in the case of an image to be subjected to inter coding. To do.

  The rate control unit 117 controls the quantization operation rate of the quantization unit 105 based on the compressed image stored in the storage buffer 107 so that overflow or underflow does not occur.

[Configuration of lossless encoding unit]
FIG. 3 is a block diagram illustrating a main configuration example of the lossless encoding unit 106. As shown in FIG. 3, the lossless encoding unit 106 includes a control unit 151, a storage unit 152, a slice 1 processing unit 153, and a slice 2 processing unit 154.

  The control unit 151 controls data input / output of the storage unit 152. The control unit 151 includes a macroblock detection unit 161 and a slice detection unit 162.

  The coefficient data is supplied from the quantization unit 105 for each macroblock. For example, as indicated by the solid line arrow in FIG. 1A, the coefficient data is one macroblock in order from the leftmost macroblock to the rightmost macroblock in the horizontal arrangement of macroblocks (hereinafter referred to as macroblock line). Supplied one by one. For example, as indicated by a dotted arrow in FIG. 1A, the upper macroblock line in the image is preferentially supplied.

  That is, the coefficient data of one image (one picture or one field) is supplied from the upper left macroblock in the image, and finally the lower right macroblock is supplied.

  The macroblock detection unit 161 detects each macroblock of the coefficient data supplied from the quantization unit 105 in this order, and stores the start address of each macroblock line in the storage unit 152 together with the coefficient data. Further, the macroblock detection unit 161 notifies the slice detection unit 162 of the start address of the detected macroblock line.

  The slice detection unit 162 detects the boundary of the entropy slice (the boundary between slice 1 and slice 2) from the start address of the macroblock line, and operates the slice 1 processing unit 153 and the slice 154 according to the detection result. .

  For example, as shown in FIG. 1A, when an image (one picture or one field) is divided into two parts, slice 1 and slice 2, the slice detector 162 determines the upper half of the image from the start address of the macroblock line. It is detected that the coefficient data is stored in the storage unit 152, and the boundary of the entropy slice is detected.

  When the slice detection unit 162 determines that the writing of coefficient data of the bottom macroblock line of slice 1 has started, the slice detection unit 162 operates the slice 1 processing unit 153 to read the coefficient data of slice 1 from the storage unit 152. Let it begin.

  If it is determined that the writing of the coefficient data of the next lower macroblock line is started, the slice detection unit 162 also operates the slice 2 processing unit 154 to read the coefficient data of slice 2 from the storage unit 152. Let it begin.

  The storage unit 152 includes an arbitrary recording medium such as a semiconductor memory such as a RAM (Random Access Memory) and a flash memory, and a hard disk, and stores information such as coefficient data and the head address of the macroblock line. Basically, the arithmetic coding process (CABAC) performed by the slice 1 processing unit 153 and the slice 2 processing unit 154 has a heavy load and a long processing time. In addition, the processing speed changes. Therefore, by buffering the coefficient data supplied to the storage unit 152, occurrence of overflow or underflow can be suppressed.

  The slice 1 processing unit 153 reads the coefficient data of slice 1 from the storage unit 152 and performs CABAC. The slice 1 processing unit 153 includes a read control unit 171, a macro block line memory 172, a context calculation unit 173, a binarization unit 174, and an arithmetic coding unit 175.

  The read control unit 171 refers to the start address of the macroblock line written in an area different from the area where the coefficient data of the storage unit 152 is stored, and reads the coefficient data of slice 1 from the storage unit 152 for each macroblock. Read to. The read control unit 171 reads the coefficient data preferentially from the bottom macroblock line of slice 1. The read control unit 171 stores the read coefficient data in the macroblock line memory 172 and supplies it to the binarization unit 174.

  The macro block line memory 172 includes an arbitrary recording medium such as a semiconductor memory such as a RAM (Random Access Memory) and a flash memory, and a hard disk, and stores coefficient data of at least one macro block line.

  The context calculation unit 173 uses the coefficient data stored in the macroblock line memory 172 to obtain the state of the neighboring macroblock adjacent to the processing target macroblock, and based on the surrounding state, the occurrence probability table and the occurrence A context (Context) indicating a symbol with a high probability is calculated and supplied to the arithmetic encoding unit 175.

  The binarization unit 174 binarizes the coefficient data (multi-value data) supplied from the read control unit 171 and supplies the binarized data (binary data) to the arithmetic encoding unit 175.

  The arithmetic encoding unit 175 performs binary arithmetic encoding on the binary data supplied from the binarizing unit 174 according to the context supplied from the context calculating unit 173. The arithmetic encoding unit 175 supplies the obtained encoded data to the accumulation buffer 107 and accumulates it.

  The slice 2 processing unit 154 mainly reads the coefficient data of slice 2 from the storage unit 152 and performs CABAC. The slice 2 processing unit 154 basically has the same configuration as the slice 1 processing unit 153. That is, the slice 2 processing unit 154 includes a read control unit 181, a macroblock line memory 182, a context calculation unit 183, a binarization unit 184, and an arithmetic coding unit 185.

  The read control unit 181 has the same configuration as the read control unit 171 and performs the same processing. The macro block line memory 182 has the same configuration as the macro block line memory 172 and performs the same processing. The context calculation unit 183 has the same configuration as the context calculation unit 173 and performs the same processing. The binarization unit 184 has the same configuration as the binarization unit 174 and performs the same processing. The arithmetic encoding unit 185 has the same configuration as the arithmetic encoding unit 175 and performs the same processing.

  However, in the case of slice 2, the processing order and reference direction of macroblocks are different from the case of slice 1.

[Encoding procedure]
FIG. 4 is a diagram for explaining an example of the processing progress direction and the reference direction of the present invention.

  CABAC for one image (one picture or one field) is performed in the order shown by the arrows in FIG. 4A. That is, the image is divided into a plurality of entropy slices (large areas) arranged in the vertical direction, and CABAC (encoding) is performed for each macroblock (small area) that divides each entropy slice in the matrix direction.

  In slice 1, the coefficient data is encoded with the bottom macroblock line (small area row) first, and then the next higher macroblock line, as shown by the dotted arrows in FIG. 4A. Is done. In this way, the processing target moves to the upper macroblock line line by line, and finally the uppermost macroblock line is encoded.

  Within each macroblock line, as shown by the solid line arrow in FIG. 4A, the leftmost macroblock is encoded first, and then the right macroblock is encoded, as in the conventional case. . In this way, the processing target moves to the right macroblock one by one, and finally the rightmost macroblock is encoded.

  The coefficient data is stored in the storage unit 152 for each macroblock in order from the upper macroblock line to the lower macroblock line in the image, as indicated by the arrow in FIG. 1A. That is, when the coefficient data of the leftmost macroblock of the bottommost macroblock line of slice 1 is written into the storage unit 152, CABAC of slice 1 is started.

  In the case of the conventional CABAC, as shown in FIG. 1C, the macroblock (mbA) one left of the processing target macroblock (curr) and the macroblock (mbB) one level above are referred to as peripheral macroblocks. . However, in the case of this slice 1, as shown in FIG. 4B, the macroblock (mbA) that is one left of the processing target macroblock (curr) and the macroblock (mbB ′) that is one lower are used as the peripheral macroblocks. Referenced.

  However, since the macroblock line immediately below the lowest macroblock line encoded first is the macroblock line of slice 2, as shown in FIG. 4C, reference is impossible. In this case, although the reference direction is different, processing is performed in the same manner as in the case of the top macroblock line of the conventional CABAC.

  Also, as shown in FIG. 4C, when the processing target is the macroblock at the left end of the screen, the macroblock (mbA) on the left side cannot be referred to. In this case, the processing is the same as in the case of the conventional CABAC.

  On the other hand, in the slice 2, as shown by the arrow in FIG. 4A, the encoding proceeds as in the conventional case. That is, in the coefficient data, as indicated by a dotted arrow in FIG. 4A, the uppermost macroblock line is encoded first, and then the next lower macroblock line is encoded. In this way, the processing target moves to the lower macroblock line one by one, and finally the lowermost macroblock line is encoded.

  Within each macroblock line, as shown by the solid arrows in FIG. 4A, the leftmost macroblock is first encoded, and then the rightmost macroblock is encoded. In this way, the processing target moves to the right macroblock one by one, and finally the rightmost macroblock is encoded.

  In the case of this slice 2, as shown in FIG. 4D, the reference direction of the neighboring macroblocks is the same as in the conventional case, and the macroblock (mbA) one left of the processing target macroblock (curr) and 1 The upper macroblock (mbB) is referred to as the peripheral macroblock.

  Therefore, the CABAC of slice 2 can be started when the coefficient data of the leftmost macroblock of the topmost macroblock line of slice 2 is written in the storage unit 152. That is, the CABAC of slice 2 can be started without waiting for the completion of CABAC of slice 1. For example, CABAC of slice 2 can be executed in parallel with CABAC of slice 1.

  The CABAC of slice 1 does not refer to the coefficient data of slice 2. As will be described later, for the top macroblock line of slice 2, the bottom macroblock line of slice 1 is referred to, but the coefficient data of slice 1 starts CABAC of slice 2 Is present in the storage unit 152. Accordingly, the CABAC of slice 1 and the CABAC of slice 2 can proceed independently of each other without requiring processing waiting or the like.

  Thereby, the lossless encoding part 106 can improve the throughput of an encoding process. In other words, the lossless encoding unit 106 can perform encoding at higher speed.

  In addition, as described above, the macroblock line on the top that is referred to at the time of CABAC of the top macroblock line in slice 2 is the bottom macroblock line in slice 1 and has already been stored in the storage unit. 152 is stored. Therefore, as shown in FIG. 4D, in the case of slice 2 CABAC, refer to the upper macroblock (mbB) in the encoding of the first macroblock line, ie, the topmost macroblock line. Can do.

  More specifically, the read control unit 181 of the slice 2 processing unit 154 reads the lowermost macroblock line of the slice 1 before reading the coefficient data of the uppermost macroblock line of the slice 2 as a processing target. Read and hold in the macroblock line memory 182.

  The context calculation unit 183 performs one of the slices 1 stored in the macroblock line memory 182 for the arithmetic coding process performed on the coefficient data of the top macroblock line of the slice 2 performed by the arithmetic coding unit 185. The peripheral macroblock context is calculated and provided using coefficient data of the bottom macroblock line.

  The arithmetic encoding unit 185 performs arithmetic encoding processing on the coefficient data of the top macroblock line of slice 2 using the context.

  As described above, since the neighboring macroblocks can be referenced even at the entropy slice boundary, the lossless encoding unit 106 can suppress a reduction in encoding efficiency due to image division (multi-slicing).

[Encoding process]
Next, the flow of each process executed by the image encoding device 100 as described above will be described. First, an example of the flow of the encoding process will be described with reference to the flowchart of FIG.

  In step S101, the A / D conversion unit 101 performs A / D conversion on the input image. In step S102, the screen rearrangement buffer 102 stores the image supplied from the A / D conversion unit 101, and rearranges the picture from the display order to the encoding order.

  In step S103, the intra prediction unit 114 and the motion prediction / compensation unit 115 each perform image prediction processing. That is, in step S103, the intra prediction unit 114 performs an intra prediction process in the intra prediction mode. The motion prediction / compensation unit 115 performs motion prediction / compensation processing in the inter prediction mode.

  In step S104, the selection unit 116 determines the optimal prediction mode based on the cost function values output from the intra prediction unit 114 and the motion prediction compensation unit 115. That is, the selection unit 116 selects either the prediction image generated by the intra prediction unit 114 or the prediction image generated by the motion prediction / compensation unit 115.

  The prediction image selection information is supplied to the intra prediction unit 114 or the motion prediction / compensation unit 115. When the prediction image of the optimal intra prediction mode is selected, the intra prediction unit 114 supplies information indicating the optimal intra prediction mode (that is, intra prediction mode information) to the lossless encoding unit 106.

  When the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 115 outputs information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode to the lossless encoding unit 106. To do. Information according to the optimal inter prediction mode includes motion vector information, flag information, reference frame information, and the like.

  In step S105, the calculation unit 103 calculates a difference between the image rearranged in step S102 and the predicted image obtained by the prediction process in step S103. The predicted image is supplied from the motion prediction / compensation unit 115 in the case of inter prediction and from the intra prediction unit 114 in the case of intra prediction to the calculation unit 103 via the selection unit 116.

  The data amount of the difference data is reduced compared to the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

  In step S <b> 106, the orthogonal transform unit 104 performs orthogonal transform on the difference information supplied from the calculation unit 103. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed, and transformation coefficients are output. In step S107, the quantization unit 105 quantizes the transform coefficient.

  In step S108, the lossless encoding unit 106 encodes the quantized transform coefficient output from the quantization unit 105. That is, lossless encoding such as arithmetic encoding is performed on the difference image (secondary difference image in the case of Inter). Details of the encoding process will be described later.

  The lossless encoding unit 106 encodes information related to the prediction mode of the prediction image selected by the process of step S104, and adds the information to the header information of the encoded data obtained by encoding the difference image.

  That is, the lossless encoding unit 106 encodes the intra prediction mode information supplied from the intra prediction unit 114 or the information corresponding to the optimal inter prediction mode supplied from the motion prediction compensation unit 115, and adds the information to the header information. To do.

  In step S109, the accumulation buffer 107 accumulates encoded data output from the lossless encoding unit 106. The encoded data stored in the storage buffer 107 is appropriately read out and transmitted to the decoding side via the transmission path.

  In step S <b> 110, the rate control unit 117 controls the quantization operation rate of the quantization unit 105 based on the compressed image stored in the storage buffer 107 so that overflow or underflow does not occur.

  Further, the difference information quantized by the process of step S107 is locally decoded as follows. That is, in step S <b> 111, the inverse quantization unit 108 inversely quantizes the transform coefficient quantized by the quantization unit 105 with characteristics corresponding to the characteristics of the quantization unit 105. In step S <b> 112, the inverse orthogonal transform unit 109 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 108 with characteristics corresponding to the characteristics of the orthogonal transform unit 104.

  In step S113, the calculation unit 110 adds the predicted image input via the selection unit 116 to the locally decoded difference information, and corresponds to the locally decoded image (corresponding to the input to the calculation unit 103). Image). In step S <b> 114, the deblocking filter 111 filters the image output from the calculation unit 110. Thereby, block distortion is removed. In step S115, the frame memory 112 stores the filtered image. It should be noted that an image that has not been filtered by the deblocking filter 111 is also supplied from the computing unit 110 and stored in the frame memory 112.

[Lossless encoding process]
Next, an example of the flow of lossless encoding processing executed in step S108 of FIG. 5 will be described with reference to the flowchart of FIG.

  When the lossless encoding process is started, in step S131, the storage unit 152 stores the coefficient data for each macroblock supplied via the macroblock detection unit 161. In step S132, the storage unit 152 stores the head address of the macroblock line detected by the macroblock detection unit 161 in a region different from the coefficient data.

  In step S133, the slice detection unit 162 determines whether or not the macroblock stored in the storage unit 152 is a macroblock at the boundary of the entropy slice. If it is determined that the macroblock is at the boundary of the entropy slice, the process is performed. Proceed to step S134.

  In step S134, the slice detection unit 162 controls the slice 1 processing unit 153 at the time when the leftmost macroblock of the bottom macroblock line of slice 1 is written in the storage unit 152. Encoding from below to above (slice 1 encoding processing described later) is started. That is, the slice detection unit 162 causes the slice 1 processing unit 153 to execute CABAC as another task in the order from the bottom to the top of the slice 1 as described with reference to FIG.

  In step S <b> 135, the slice detection unit 162 controls the slice 2 processing unit 154 when the leftmost macroblock of the topmost macroblock line of slice 2 is written in the storage unit 152. Encoding from above to below (slice 2 encoding processing described later) is started. That is, the slice detection unit 162 causes the slice 2 processing unit 154 to execute CABAC as another task in the order from the top to the bottom of the slice 2 as described with reference to FIG.

  When CABAC for slice 1 and CABAC for slice 2 are started, the slice detector 162 advances the process to step S136. When it is determined in step S133 that the boundary is not a slice boundary, the slice detection unit 162 advances the process to step S136.

  In step S136, the macroblock detection unit 161 determines whether all the macroblocks in the image (picture or field) have been processed. If it is determined that there is an unprocessed macroblock, the process is performed in step S131. And the subsequent processing is repeated for the next supplied macroblock.

  If it is determined in step S136 that all macroblocks have been processed, the macroblock detection unit 161 ends the lossless encoding process, returns the process to step S108 in FIG. 5, and advances the process to step S109. .

[Slice 1 encoding process]
Next, an example of the flow of slice 1 encoding processing executed by the slice 1 processing unit 153 will be described with reference to the flowchart of FIG.

  When the process of step S134 of FIG. 6 is executed and the slice 1 encoding process is started, the read control unit 171 of the slice 1 processing unit 153 obtains the coefficient data of the last macroblock line of the slice 1 in step S151. read out. In step S152, the macro block line memory 172 stores the coefficient data of the macro block line read in step S151.

  In step S153, the binarizing unit 174 binarizes the coefficient data of the macroblock line read in step S151. In step S154, the arithmetic encoding unit 175 performs arithmetic encoding without referring to the lower macroblock line.

  When all the coefficient data of the bottom macroblock line of slice 1 is encoded as described above, slice 1 processing section 153 advances the processing to step S155. In step S155, the read control unit 171 of the slice 1 processing unit 153 reads the coefficient data of the macroblock line that is one layer above the macroblock line read last time. In step S156, the macro block line memory 172 stores the coefficient data of the macro block line read in step S155.

  In step S157, the binarization unit 174 binarizes the coefficient data of the macroblock line read in step S155. In step S158, the context calculation unit 173 calculates the context of the processing target macroblock with reference to the next lower macroblock line. In step S159, the arithmetic encoding unit 175 performs arithmetic encoding using the context calculated in step S158.

  In step S160, the read control unit 171 determines whether or not all macroblock lines in the image (picture or field) have been processed. If there is an unprocessed macroblock line, the process returns to step S155. Repeat the subsequent processing.

  When the processes in steps S155 to S160 are repeatedly executed and it is determined in step S160 that all macroblock lines in the image have been processed, the read control unit 171 ends the slice 1 encoding process.

[Slice 2 encoding process]
Next, an example of the flow of slice 2 encoding processing executed by the slice 2 processing unit 154 will be described with reference to the flowchart of FIG.

  When the process of step S135 in FIG. 6 is executed and the slice 2 encoding process is started, the read control unit 181 of the slice 2 processing unit 154 reads the coefficient data of the last macroblock line of slice 1 in step S181. The coefficient data of the first macroblock line of slice 2 is read out. In step S182, the macro block line memory 182 stores the coefficient data of the macro block line read in step S181.

  In step S183, the binarizing unit 184 binarizes the coefficient data of the first macroblock line of slice 2 read in step S181. In step S184, the context calculation unit 183 refers to the last macroblock line of slice 1 stored in step S182, and calculates the context of the processing target macroblock. In step S185, the arithmetic encoding unit 185 performs arithmetic encoding using the context calculated in step S184.

  As described above, when all the coefficient data of the top macroblock line of slice 2 are encoded, slice 2 processing section 154 advances the processing to step S186. In step S186, the read control unit 181 of the slice 2 processing unit 154 reads the coefficient data of the macroblock line immediately below the macroblock line read last time. In step S187, the macroblock line memory 172 stores the coefficient data of the macroblock line read in step S186.

  In step S188, the binarizing unit 184 binarizes the coefficient data of the macroblock line read in step S186. In step S189, the context calculation unit 183 calculates the context of the processing target macroblock with reference to the upper macroblock line. In step S190, the arithmetic encoding unit 185 performs arithmetic encoding using the context calculated in step S189.

  In step S191, the read control unit 181 determines whether all the macroblock lines in the image (picture or field) have been processed. If there is an unprocessed macroblock line, the process returns to step S186. Repeat the subsequent processing.

  When the processes in steps S186 to S191 are repeatedly executed and it is determined in step S191 that all macroblock lines in the image have been processed, the read control unit 181 ends the slice 2 encoding process.

  As described above, the lossless encoding unit 106 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

<2. Second Embodiment>
[Image decoding device]
The encoded data encoded by the image encoding device 100 described in the first embodiment is transmitted to an image decoding device corresponding to the image encoding device 100 via a predetermined transmission path and decoded. .

  Hereinafter, the image decoding apparatus will be described. FIG. 9 is a block diagram illustrating a main configuration example of an image decoding device to which the present invention has been applied.

  As shown in FIG. 9, the image decoding apparatus 200 includes a storage buffer 201, a lossless decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transform unit 204, a calculation unit 205, a deblock filter 206, a screen rearrangement buffer 207, A D / A conversion unit 208, a frame memory 209, a selection unit 210, an intra prediction unit 211, a motion prediction compensation unit 212, and a selection unit 213 are included.

  The accumulation buffer 201 accumulates the transmitted encoded data. This encoded data is encoded by the image encoding device 100. The lossless decoding unit 202 decodes the encoded data read from the accumulation buffer 201 at a predetermined timing by a method corresponding to the encoding method of the lossless encoding unit 106 in FIG.

  The inverse quantization unit 203 inversely quantizes the coefficient data obtained by decoding by the lossless decoding unit 202 by a method corresponding to the quantization method of the quantization unit 105 in FIG. The inverse quantization unit 203 supplies the inversely quantized coefficient data to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 204 is a method corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 2, performs inverse orthogonal transform on the coefficient data, and corresponds to the residual data before being orthogonally transformed in the image encoding device 100. Decoding residual data to be obtained is obtained.

  Decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 205. Further, a prediction image is supplied from the intra prediction unit 211 or the motion prediction compensation unit 212 to the calculation unit 205 via the selection unit 213.

  The computing unit 205 adds the decoded residual data and the predicted image, and obtains decoded image data corresponding to the image data before the predicted image is subtracted by the computing unit 103 of the image encoding device 100. The arithmetic unit 205 supplies the decoded image data to the deblock filter 206.

  The deblocking filter 206 removes the block distortion of the decoded image, and then supplies it to the frame memory 209 for storage, and also supplies it to the screen rearrangement buffer 207.

  The screen rearrangement buffer 207 rearranges images. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 102 in FIG. 2 is rearranged in the original display order. The D / A conversion unit 208 D / A converts the image supplied from the screen rearrangement buffer 207, outputs it to a display (not shown), and displays it.

  The selection unit 210 reads out the inter-processed image and the referenced image from the frame memory 209 and supplies them to the motion prediction / compensation unit 212. Further, the selection unit 210 reads an image used for intra prediction from the frame memory 209 and supplies the image to the intra prediction unit 211.

  Information indicating the intra prediction mode obtained by decoding the header information is appropriately supplied from the lossless decoding unit 202 to the intra prediction unit 211. The intra prediction unit 211 generates a predicted image based on this information, and supplies the generated predicted image to the selection unit 213.

  The motion prediction / compensation unit 212 acquires information (prediction mode information, motion vector information, reference frame information) obtained by decoding the header information from the lossless decoding unit 202. When the information indicating the inter prediction mode is supplied, the motion prediction / compensation unit 212 generates a prediction image based on the inter motion vector information from the lossless decoding unit 202, and supplies the generated prediction image to the selection unit 213. .

  The selection unit 213 selects the prediction image generated by the motion prediction / compensation unit 212 or the intra prediction unit 211 and supplies the selected prediction image to the calculation unit 205.

[Reversible decoding unit]
FIG. 10 is a block diagram illustrating a main configuration example of the lossless decoding unit 202 of FIG.

  As shown in FIG. 10, the lossless decoding unit 202 includes a demultiplexer 251, a slice 1 processing unit 252, a slice 2 processing unit 253, a storage unit 254, and a read control unit 255.

  The demultiplexer 251 identifies the entropy slice (slice 1 or slice 2) to which the encoded data supplied from the accumulation buffer 201 belongs, and controls the supply destination of the encoded data by the entropy slice.

  The bit stream generated by the image encoding device 100 in FIG. 2 is multiplexed as an independent stream for each entropy slice. The multiplexed stream is supplied from the accumulation buffer 201.

  For example, the demultiplexer 251 supplies the encoded data belonging to the slice 1 to the slice 1 processing unit 252. For example, in the case of encoded data belonging to slice 2, the demultiplexer 251 supplies the slice 2 processing unit 253 with it.

  The slice 1 processing unit 252 performs arithmetic decoding on the encoded data belonging to the slice 1. In the image encoding device 100, each macroblock of slice 1 is encoded in the order from left to right and from bottom to top of the image. The peripheral blocks to be referred to are a macroblock one left of the processing target macroblock and a macroblock one below. Decoding processing is also performed in the same processing order and reference direction.

  The slice 2 processing unit 253 performs arithmetic decoding on the encoded data belonging to the slice 2. In the image encoding device 100, each macroblock of slice 2 is encoded in the order from left to right and from top to bottom of the image. In addition, the peripheral blocks to be referred to are a macroblock one left of the processing target macroblock and a macroblock one above. Decoding processing is also performed in the same processing order and reference direction.

  The storage unit 254 acquires and stores the decoding coefficient data (multilevel data) generated by the slice 1 processing unit 252 and the decoding coefficient data (multilevel data) generated by the slice 2 processing unit 253. The read control unit 255 reads the decoded coefficient data stored in the storage unit 254 in a predetermined order at a predetermined timing, and supplies the decoded coefficient data to the inverse quantization unit 203.

  Actually, the lossless decoding unit 202 extracts metadata such as encoding parameters and prediction mode information from the supplied bitstream, and supplies them to the intra prediction unit 211 and the motion prediction compensation unit 212.

  The slice 1 processing unit 252 includes an arithmetic decoding unit 261, a multi-value conversion unit 262, a macroblock line memory 263, and a context calculation unit 264.

  The arithmetic decoding unit 261 performs arithmetic decoding on the encoded data using the context calculated by the context calculation unit 264, and generates binary data. The multi-value conversion unit 262 multi-values the binary data output from the arithmetic decoding unit 261. The multi-value conversion unit 262 supplies the generated multi-value data to the storage unit 254 as decoding coefficient data and also supplies it to the macroblock line memory 263.

  The macro block line memory 263 stores the decoding coefficient data supplied from the multi-value quantization unit 262. The macro block line memory 263 can store one or more macro line blocks of decoding coefficient data.

  The context calculation unit 264 uses the decoding coefficient data stored in the macroblock line memory 263 to calculate the context of a neighboring macroblock adjacent to the processing target macroblock of the decoding process by the arithmetic decoding unit 261, and calculates the arithmetic The data is supplied to the decoding unit 261.

  For example, the context calculation unit 264 generates the context of the macroblock one left of the processing target macroblock and the context of the macroblock one level lower than the processing target macroblock.

  The slice 2 processing unit 253 includes an arithmetic decoding unit 271, a multi-value conversion unit 272, a macroblock line memory 273, and a context calculation unit 274.

  The arithmetic decoding unit 271 arithmetically decodes the encoded data using the context calculated by the context calculation unit 274 to generate binary data. The multi-value conversion unit 272 multi-values the binary data output from the arithmetic decoding unit 271. The multi-value conversion unit 272 supplies the generated multi-value data to the storage unit 254 as decoding coefficient data and also supplies it to the macroblock line memory 273.

  The macro block line memory 273 stores the decoding coefficient data supplied from the multi-value quantization unit 272. The macroblock line memory 273 can store the decoding coefficient data for one macroline block or more.

  The context calculation unit 274 uses the decoding coefficient data stored in the macroblock line memory 273 to calculate the context of the neighboring macroblock adjacent to the processing target macroblock of the decoding process by the arithmetic decoding unit 271, and performs arithmetic The data is supplied to the decoding unit 271.

  For example, the context calculation unit 274 generates the context of the macroblock one left of the processing target macroblock and the context of the macroblock immediately above the processing target macroblock.

  When the decoding target is the top macroblock line of slice 2, the context calculation unit 274 stores the bottommost macro of slice 1 stored in the macroblock line memory 263 of the slice 1 processing unit 252. Using the decoding coefficient data of the block line, a context of the macro block one level above the processing target macro block is generated.

  By performing the decoding process as described above, the lossless decoding unit 202 proceeds in the same order as in the case of the lossless encoding unit 106 and performs peripheral reference in the same direction, so that the encoded data is correctly decoded. be able to.

[Decryption process]
Next, the flow of each process executed by the image decoding apparatus 200 as described above will be described. First, an example of the flow of decoding processing will be described with reference to the flowchart of FIG.

  When the decoding process is started, in step S201, the accumulation buffer 201 accumulates the transmitted encoded data. In step S202, the lossless decoding unit 202 decodes the encoded data supplied from the accumulation buffer 201. That is, the I picture, P picture, and B picture encoded by the lossless encoding unit 106 in FIG. 2 are decoded.

  At this time, motion vector information, reference frame information, prediction mode information (intra prediction mode or inter prediction mode), flag information, and the like are also decoded.

  That is, when the prediction mode information is intra prediction mode information, the prediction mode information is supplied to the intra prediction unit 211. When the prediction mode information is inter prediction mode information, motion vector information corresponding to the prediction mode information is supplied to the motion prediction / compensation unit 212.

  In step S203, the inverse quantization unit 203 inversely quantizes the transform coefficient decoded by the lossless decoding unit 202 with characteristics corresponding to the characteristics of the quantization unit 105 in FIG. In step S204, the inverse orthogonal transform unit 204 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 203 with characteristics corresponding to the characteristics of the orthogonal transform unit 104 in FIG. As a result, the difference information corresponding to the input of the orthogonal transform unit 104 (output of the calculation unit 103) in FIG. 2 is decoded.

  In step S <b> 205, the intra prediction unit 211 or motion prediction / compensation unit 212 performs image prediction processing corresponding to the prediction mode information supplied from the lossless decoding unit 202.

  That is, when intra prediction mode information is supplied from the lossless decoding unit 202, the intra prediction unit 211 performs intra prediction processing in the intra prediction mode. When inter prediction mode information is supplied from the lossless decoding unit 202, the motion prediction / compensation unit 212 performs motion prediction processing in the inter prediction mode.

  In step S206, the selection unit 213 selects a predicted image. That is, the prediction image generated by the intra prediction unit 211 or the prediction image generated by the motion prediction compensation unit 212 is supplied to the selection unit 213. The selection unit 213 selects one of them. The selected prediction image is supplied to the calculation unit 205.

  In step S207, the calculation unit 205 adds the predicted image selected by the process of step S206 to the difference information obtained by the process of step S204. As a result, the original image data is decoded.

  In step S208, the deblocking filter 206 filters the decoded image data supplied from the calculation unit 205. Thereby, block distortion is removed.

  In step S209, the frame memory 209 stores the filtered decoded image data.

  In step S210, the screen rearrangement buffer 207 rearranges the frames of the decoded image data. That is, the order of frames of the decoded image data rearranged for encoding by the screen rearrangement buffer 102 (FIG. 2) of the image encoding device 100 is rearranged to the original display order.

  In step S211, the D / A conversion unit 208 performs D / A conversion on the decoded image data in which the frames are rearranged in the screen rearrangement buffer 207. The decoded image data is output to a display (not shown), and the image is displayed.

[Reversible decoding process]
Next, an example of the flow of lossless decoding processing executed in step S202 of FIG. 11 will be described with reference to the flowchart of FIG.

  When the lossless decoding process is started, the demultiplexer 251 divides the coefficient data for each slice in step S231. In step S232, the demultiplexer 251 supplies the encoded data of slice 1 to the slice 1 processing unit 252, and starts decoding from the lower side of slice 1 to the upper side (slice 1 decoding process) as another task.

  In step S233, the demultiplexer 251 determines whether or not the last (bottom) macroblock line of slice 1 has been decoded by the slice 1 decoding process started in step S232, and determines that the decoding has ended. Wait until If it is determined that decoding of the last (bottom) macroblock line of slice 1 has been completed, the demultiplexer 251 advances the process to step S234.

  In step S234, the demultiplexer 251 supplies the encoded data of slice 2 to the slice 2 processing unit 253, and starts decoding from the upper side to the lower side of slice 2 (slice 2 decoding process) as another task.

  In step S235, the demultiplexer 251 determines whether or not decoding of all macroblocks in the slice 2 has been completed by the slice 2 decoding process started in step S234, and waits until it is determined that the processing has been completed. If it is determined that the decoding of slice 2 has ended, the demultiplexer 251 ends the lossless decoding process, returns the process to step S202 of FIG. 11, and advances the process to step S203.

[Slice 1 decoding process]
Next, an example of the flow of the slice 1 decoding process started by the process of step S232 of FIG. 12 will be described with reference to the flowchart of FIG.

  When the slice 1 decoding process is started, the arithmetic decoding unit 261 calculates a macroblock of the lower macroblock line for each macroblock of the last (bottom) macroblock line of slice 1 in step S251. Performs arithmetic decoding without reference.

  In step S252, the multi-value conversion unit 262 multi-values the binary coefficient data obtained by decoding in step S252, and converts the multi-value data into coefficient data.

  In step S253, the macro block line memory 263 stores the coefficient data (multi-value data) of the processing target macro block line generated as described above. When the coefficient data of the bottom macroblock line of slice 1 is stored, the macroblock line memory 263 advances the process to step S254.

  When the process proceeds to step S254, the decoding target macroblock line moves up by one. In step S254, the context calculation unit 264 refers to not only the macro block line to be decoded but also the macro block line immediately below the macro block line to be decoded, and calculates the context.

  In step S255, the arithmetic decoding unit 261 performs arithmetic decoding using the context generated in step S254. In step S256, the multi-value conversion unit 262 multi-values the binary coefficient data obtained by decoding in step S255, and generates multi-value data.

  In step S257, the macro block line memory 263 stores the coefficient data (multi-value data) of the processing target macro block line generated as described above. After storing the coefficient data of the macro block line to be processed, the macro block line memory 263 advances the process to step S258.

  In step S258, the slice 1 processing unit 252 determines whether all the macroblock lines in the slice 1 have been processed. When it is determined that there is an unprocessed macroblock line, the slice 1 processing unit 252 returns the process to step S254, sets the macroblock line one above the macroblock line that was the previous processing target as the processing target, Repeat the process.

  The processes in steps S254 to S258 are repeated, and if it is determined in step S258 that all macroblock lines have been processed, the slice 1 decoding process is terminated.

[Slice 2 decoding process]
Next, an example of the flow of the slice 2 decoding process started by the process of step S234 of FIG. 12 will be described with reference to the flowchart of FIG.

  When the slice 2 decoding process is started, in step S271, the context calculation unit 274, for each macroblock of the first (top) macroblock line of the slice 2 that is the processing target, The context is calculated by referring not only to the line but also to the last (bottom) macroblock line of slice 1 stored in the macroblock line memory 263.

  In step S272, the arithmetic decoding unit 271 performs arithmetic decoding using the context calculated in step S271. In step S273, the multi-value conversion unit 272 multi-values the binary coefficient data obtained by decoding in step S272, and converts the multi-value data into coefficient data.

  In step S274, the macro block line memory 273 stores the coefficient data (multi-value data) of the processing target macro block line generated as described above. When the coefficient data of the top macroblock line of slice 2 is stored, the macroblock line memory 273 advances the process to step S275.

  When the process proceeds to step S275, the macroblock line to be decoded is moved down by one. In step S275, the context calculation unit 274 refers to not only the decoding target macroblock line but also the macroblock line immediately above the decoding target macroblock line, and calculates the context.

  In step S276, the arithmetic decoding unit 271 performs arithmetic decoding using the context generated in step S275. In step S277, the multi-value conversion unit 272 multi-values the binary coefficient data obtained by decoding in step S276 to generate multi-value data.

  In step S278, the macro block line memory 273 stores the coefficient data (multi-value data) of the processing target macro block line generated as described above. When the coefficient data of the macro block line to be processed is stored, the macro block line memory 273 advances the process to step S279.

  In step S279, the slice 2 processing unit 253 determines whether all the macroblock lines in the slice 2 have been processed. When it is determined that there is an unprocessed macroblock line, the slice 2 processing unit 253 returns the process to step S275, sets the macroblock line immediately below the macroblock line that was the previous processing target as the processing target, Repeat the process.

  The processes from step S275 to step S279 are repeated, and if it is determined in step S279 that all macroblock lines have been processed, the slice 2 decoding process is terminated.

  As described above, the image decoding apparatus 200 can correctly decode the encoded data obtained by the encoding by the image encoding apparatus 100. Thereby, the lossless decoding unit 202 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

<3. Third Embodiment>
[Configuration of lossless encoding unit]
In the above description, the context is generated by the slice 1 processing unit 153 and the slice 2 processing unit 154. However, the present invention is not limited to this, and the context may be generated before storing in the storage unit 152. Good.

  FIG. 15 is a block diagram illustrating another configuration example of the lossless encoding unit 106 of FIG.

  As shown in FIG. 15, in this case, the lossless encoding unit 106 includes a context processing unit 301 in addition to the configuration in the case of FIG. Further, the lossless encoding unit 106 includes a slice 1 processing unit 303 instead of the slice 1 processing unit 153 in the case of FIG. 3, and further includes a slice 2 processing unit 304 instead of the slice 2 processing unit 154.

  The context processing unit 301 calculates a context from the coefficient data supplied from the quantization unit 105. The slice 1 processing unit 303 basically has the same configuration as that of the slice 1 processing unit 153, but since the context processing unit 301 exists outside the slice 1 processing unit 303, the macroblock line memory 172 and the context calculation unit 173 Is omitted. The slice 2 processing unit 304 basically has the same configuration as the slice 2 processing unit 154, but the context processing unit 301 exists outside the slice 2 processing unit 304, so that the macroblock line memory 182 and the context calculation unit 183 are included. Is omitted.

  The context processing unit 301 includes a macro block line memory 311 and a context calculation unit 312. The macro block line memory 311 is basically the same as the macro block line memory 172 and the macro block line memory 182, and stores coefficient data supplied from the quantization unit 105. Note that the reference direction of the slice 1 is opposite to the conventional one, and the macro block line immediately below the macro block to be processed is referred to. Therefore, the macro block line memory 311 stores all the coefficient data of at least one macro block line. You need to be able to remember.

  The context calculation unit 312 is basically the same as the context calculation unit 173 and the context calculation unit 183, and calculates a context by referring to a macroblock line stored in the macroblock line memory 311.

  When calculating the context in slice 2, the context calculation unit 312 refers to the macro block line that is one step above the processing target. However, when calculating the context in slice 1, the context calculation unit 312 selects the macro block line that is one step below the processing target. refer.

  In other words, when calculating the context in slice 2, the context calculation unit 312 and the macroblock one mb-A left one and the macroblock one higher than the processing target macroblock (curr), as in the past. Context (context index) is calculated with reference to (mb-B).

  On the other hand, when the context calculation unit 312 calculates the context in slice 1, the macro block (mb-A) that is one left and the macro block (mb that is one lower than the processing target macro block (curr)) (mb) -B ') is referred to and a context (context index) is calculated. In other words, the context calculation unit 312 receives the macro block to be processed (curr) stored in the macro block line memory 311 and the one macro to the left when the next lower macro block (mb-B ′) is supplied. The block (mb-A) is read and the context (context index) is calculated.

  Note that the context calculation unit 312 also generates the context of the bottom macroblock line of the slice 1 when generating the context with the second macroblock line from the bottom of the slice 1 as a processing target (curr).

  The context calculation unit 312 attaches a tag indicating the correspondence between the calculated context to the coefficient data and the start address of the macroblock address in an area different from the coefficient data and the start address of the macroblock address in the storage unit 152. Remember.

  The read control unit 171 of the slice 1 processing unit 303 reads the necessary context from the storage unit 152 together with the coefficient data and the head address of the macroblock address. The read control unit 171 supplies the read context to the arithmetic coding unit 175.

  The read control unit 181 of the slice 2 processing unit 304 reads the necessary context from the storage unit 152 together with the coefficient data and the head address of the macroblock address. The read control unit 181 supplies the read context to the arithmetic coding unit 185.

[Lossless encoding process]
An example of the flow of lossless encoding processing in this case will be described with reference to the flowchart of FIG. This flowchart corresponds to the flowchart of FIG.

  In this case, each process of step S301 and step S302 is performed similarly to each process of step S131 and step S132 of FIG.

  In step S303, the context processing unit 301 determines whether or not the processing target macroblock line is slice 1. If it is determined that it is slice 1, the process proceeds to step S304.

  In step S304, the context processing unit 301 calculates a context with reference to the next lower macroblock line. Note that the contexts of the top macroblock line and the bottom macroblock line of slice 1 are generated as described above. When the context is generated, the context processing unit 301 advances the process to step S306.

  If it is determined in step S303 that the processing target macroblock line is slice 2, the context processing unit 301 advances the process to step S305.

  In step S305, the context processing unit 301 calculates a context with reference to the next macroblock line. Note that the context of the top macroblock line in slice 2 is generated with reference to the bottom macroblock line in slice 1. When the context is generated, the context processing unit 301 advances the process to step S306.

  In step S306, the storage unit 152 stores the context calculated in step S304 or step S305.

  Each process of step S307 thru | or step S310 is performed similarly to each process of step S133 thru | or step S136 of FIG.

[Slice 1 encoding process]
Next, an example of the flow of the slice 1 encoding process started by the process of step S308 of FIG. 16 will be described with reference to the flowchart of FIG. This flowchart corresponds to the flowchart of FIG.

  In this case, the process is basically performed in the same manner as in FIG. 7, but since the context has already been generated, the processes in steps S152, S156, and S158 in FIG. 7 are omitted and correspond to step S151. Step S331, Step S332 corresponding to Step S153, Step S333 corresponding to Step S155, Step S334 corresponding to Step S155, Step S335 corresponding to Step S157, Step S336 corresponding to Step S159, and Step S160 Each process of step S337 is executed.

  However, in step S334, the read control unit 171 also reads the corresponding context from the storage unit 152 together with the macroblock line one slice above the slice 1. In step S336, the arithmetic encoding unit 175 performs arithmetic encoding using the context read in step S334.

[Slice 2 encoding process]
Next, an example of the flow of the slice 2 encoding process started by the process of step S309 in FIG. 16 will be described with reference to the flowchart in FIG. This flowchart corresponds to the flowchart of FIG.

  In this case as well, the process is basically performed in the same manner as in FIG. 8, but since the context has already been generated, the processes in step S182, step S184, step S187, and step S189 in FIG. Step S351 corresponding to S181, Step S352 corresponding to Step S183, Step S353 corresponding to Step S185, Step S354 corresponding to Step S186, Step S355 corresponding to Step S188, Step S356 corresponding to Step S190, and Step Each process of step S357 corresponding to S191 is performed.

  However, in step S351, the read control unit 181 reads the corresponding context from the storage unit 152 together with the first (top) macroblock line of the slice 2. That is, the context of the last (bottom) macroblock line of slice 1 is also read. In step S353, the arithmetic encoding unit 185 performs arithmetic encoding using the context read in step S351.

  Similarly, in step S <b> 354, the read control unit 181 reads the corresponding context from the storage unit 152 together with the macroblock line immediately below slice 2. In step S356, the arithmetic encoding unit 185 performs arithmetic encoding using the context read in step S354.

  As described above, the context is calculated before the coefficient data is stored in the storage unit 152, thereby reducing the load on the arithmetic coding processing (the slice 1 processing unit 303 and the slice 2 processing unit 304) after reading. be able to. In general, the load of the arithmetic coding process is large compared to the other processes before and after, and as described above, by calculating the context in advance before storing the coefficient data in the storage unit 152, The load can be distributed, and the cost can be reduced and the encoding process can be further speeded up.

[Replication of probability table]
Furthermore, the initial state of arithmetic coding can be improved.

  In the AVC standard, the CABAC probability table is initialized using parameters cabac_init_idc and parameters slice_type and SliceQPy.

  On the other hand, as shown in FIG. 19, when the processing of the bottom macroblock line of slice 1 is finished, the contents of the probability table possessed by slice 1 processing unit 153 are stored in slice 2 processing unit 154. Copy to the probability table you have. The slice 2 processing unit 154 starts CABAC using the value used by the slice 1 processing unit 153 as an initial value.

  By doing in this way, the lossless encoding part 106 can start encoding with the probability value suitable for an image rather than the initial value by a specification, and can improve encoding efficiency.

[Slice 1 encoding process]
An example of the flow of the slice 1 encoding process in this case will be described with reference to the flowchart of FIG. The flowchart shown in FIG. 20 corresponds to the flowchart of FIG.

  In step S401, the arithmetic encoding unit 175 initializes the probability table. Each process of step S402 thru | or step S405 is performed similarly to each process of step S151 thru | or step S154 of FIG.

  In step S <b> 406, the arithmetic encoding unit 175 copies the probability table of the last (rightmost) macroblock of the bottom macroblock line and supplies it to the arithmetic encoding unit 185.

  Steps S407 through S412 are performed in the same manner as steps S155 through S160 in FIG.

[Slice 2 encoding process]
An example of the flow of the slice 2 encoding process in this case will be described with reference to the flowchart of FIG. The flowchart shown in FIG. 21 corresponds to the flowchart of FIG.

  Each process of step S431 and step S432 is performed similarly to each process of step S181 and step S182 of FIG.

  In step S433, the arithmetic encoding unit 185 obtains a copy of the probability table for slice 1. Each process of step S434 and step S435 is performed similarly to each process of step S183 and step S184 of FIG.

  In step S436, the arithmetic encoding unit 185 performs arithmetic encoding using the context with the copy of the probability table of slice 1 as an initial value.

  The processes in steps S437 through S442 are performed in the same manner as the processes in steps S186 through S191 in FIG.

  Similarly, the image decoding apparatus 200 performs decoding using a copy of the probability table as an initial value. For this purpose, the image encoding device 100 provides the image decoding device 200 with information indicating whether the probability table is initialized or a copy is used, for example, by adding it to the encoded data. When information indicating that the copy of the probability table is used as an initial value is obtained, the lossless decoding unit 202 of the image decoding device 200 causes the arithmetic decoding unit 261 of the slice 1 processing unit 252 to process the bottom macroblock line of slice 1. Is completed, the contents of the CABAC probability table are copied, and the arithmetic decoding unit 271 of the slice 2 processing unit 253 starts the arithmetic decoding process using the copy. By doing in this way, the lossless decoding part 202 can use the same probability table as the case of the encoding mentioned above, and can decode encoded data correctly.

[Multi-segment]
In the above description, the image is divided into two, but the number of divisions may be other than two. For example, as shown in FIG. 22, an image may be divided into four entropy slices of slice 1 to slice 4.

  When the image is divided into four as in the example of FIG. 22, the slice 1 and the slice 2 are encoded in the same manner as in the case of the above-described two division. The slice 3 is encoded in the same manner as the slice 1, and the slice 4 is encoded in the same manner as the slice 2.

  That is, the encoding processing of slice 1 and slice 3 is advanced in the order from the lower macroblock line toward the upper macroblock line. Also, in the encoding process of slice 1 and slice 3, the macroblock one left of the processing target macroblock and the macroblock one level below are referred to as the peripheral macroblock.

  On the other hand, the encoding processing of slice 2 and slice 4 is advanced in the order from the upper macroblock line to the lower macroblock line, as in the prior art. Further, in the encoding process of slice 2 and slice 4, the macroblock one left of the processing target macroblock and the macroblock one above are referred to as the peripheral macroblock.

  Then, when the coefficient data of the lowermost macroblock line of slice 1 is stored in the storage unit 152, the lowermost macroblock line of slice 3 is started in the same manner as the encoding process of slice 2 is started. When the coefficient data is stored in the storage unit 152, the encoding process of slice 4 is started.

  Further, the encoding process of the top macroblock line of slice 4 is performed in the same manner as the bottom macroblock line of slice 1 is referred to in the encoding process of the top macroblock line of slice 2. Then, the macroblock on the bottom macroblock line of slice 3 is referred to as the macroblock one level above the processing target macroblock.

[Configuration of lossless encoding unit]
In this way, when the image is divided into four, the lossless encoding unit 106, for example, as shown in FIG. 23, the slice 3 processing unit 503 and the slice 2 processing unit 154 similar to the slice 1 processing unit 153 of FIG. And a slice 4 processing unit 504 similar to the above.

  Further, the lossless encoding unit 106 includes a control unit 501 that divides an image into four slices instead of the control unit 151 that divides the image into two slices.

  The control unit 501 includes a macroblock detection unit 161 and a slice detection unit 512. The slice detection unit 512 controls the slice 1 processing unit 153, the slice 2 processing unit 154, the slice 3 processing unit 503, and the slice 4 processing unit 504, divides the image into four slices, and performs encoding processing in parallel with each other. To do.

  Except for the image being divided into four, each process is executed in the same manner as in the case of dividing into two. That is, slice 3 is encoded in the same manner as slice 1, and slice 4 is encoded in the same manner as slice 2.

  By increasing the number of divisions in this way, the number of parallels increases, so that the throughput is improved and the lossless encoding unit 106 can perform lossless encoding processing at higher speed. At that time, since the encoding process is performed in the same manner as in the case of the two-division, the lossless encoding unit 106 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

  That is, the lossless encoding unit 106 of the image encoding device 100 divides the encoding target image into a plurality of regions arranged vertically. Then, the lossless encoding unit 106 performs encoding for the even-numbered regions from the top in the order from the top macroblock line to the bottom, as before, and the macroblock one left of the processing target macroblock as the peripheral macroblock. Refers to the block and the macroblock one level above.

  On the other hand, for the odd-numbered area from the top, the lossless encoding unit 106 performs encoding in the order from the bottom macroblock line to the top, contrary to the conventional case, and the processing target macroblock is set as the peripheral macroblock. Referring to the left macroblock and the next lower macroblock.

  Furthermore, the lossless encoding unit 106 performs encoding processing on the next lower (even number from the top) region when encoding of the bottommost macroblock line in the odd numbered region from the top is completed. To start.

  At this time, the lossless encoding unit 106 refers to the bottom macroblock line in the odd-numbered area from the top, and the top macro in the area that is one lower (even number from the top). Encoding processing for block lines is performed.

[Configuration of lossless decoding unit]
FIG. 24 shows a configuration example of the lossless decoding unit 202 of the image decoding device 200 when the image is divided into four.

  As shown in FIG. 24, the lossless decoding unit 202 in this case has a demultiplexer 551 that divides the encoded data into four for each slice, instead of the demultiplexer 251 compared to the case of FIG. Furthermore, a slice 3 processing unit 552 and a slice 4 processing unit 553 are included.

  The slice 3 processing unit 552 is a processing unit that decodes the encoded data of the slice 3 and has the same configuration as the slice 1 processing unit 252 and performs the same processing. Since the slice 3 is encoded by the same method as the slice 1, the slice 3 processing unit 552 decodes the encoded data of the slice 3 correctly by decoding the slice 3 by the same method as the slice 1 processing unit 252. can do.

  The slice 4 processing unit 553 is a processing unit that decodes the encoded data of the slice 4, has the same configuration as the slice 2 processing unit 253, and performs the same processing. Since the slice 4 is encoded by the same method as the slice 2, the slice 4 processing unit 553 decodes the encoded data of the slice 4 correctly by decoding the slice 4 by the same method as the slice 2 processing unit 253. can do.

  First, the decoding process of the encoded data of slice 1 by the slice 1 processing unit 252 and the decoding process of the encoded data of slice 3 by the slice 3 processing unit 552 are started. The decoding processing of the top macroblock line of slice 2 by the slice 2 processing unit 253 is started after waiting for the decoding processing result of the bottom macroblock line of slice 1 (referred to as a neighboring macroblock). .

  Similarly, the decoding processing of the top macroblock line of slice 4 by the slice 4 processing unit 553 is started after waiting for the decoding processing result of the bottom macroblock line of slice 3 (referred to as a peripheral macroblock). )

  That is, the lossless decoding unit 202 of the image decoding device 200 divides an image before encoding corresponding to encoded data to be decoded into a plurality of regions arranged vertically. Then, the lossless decoding unit 202 decodes the encoded data corresponding to the even-numbered regions from the top in the order from the top macroblock line to the bottom, as before, and sets one of the processing target macroblocks as a peripheral macroblock. Refer to the left macroblock and the macroblock one level above.

  On the other hand, with respect to the encoded data corresponding to the odd-numbered area from the top, the lossless decoding unit 202 decodes in order from the bottom macroblock line to the top, and processes it as a peripheral macroblock, contrary to the conventional case. Reference is made to the macroblock one to the left of the target macroblock and the macroblock one level below.

  Further, when the decoding of the encoded data corresponding to the bottommost macroblock line in the odd-numbered area from the top is completed, the lossless decoding unit 202 performs the processing of the area below that (even-numbered from the top). The decoding process for the encoded data is started.

  At this time, the lossless decoding unit 202 refers to the bottom macroblock line in the odd-numbered area from the top, and the top macroblock in the area one lower (even from the top). A decoding process is performed on the encoded data corresponding to the line.

  Note that each of the image encoding device 100 and the image decoding device 200 may have a large number of slice processing units in advance and operate the same number of slice processing units as the number of entropy slices therein. That is, the number of entropy slices may be changed in image units, sequence units, content units, or the like. In that case, the image encoding device 100 notifies the image decoding device 200 of the number of entropy slices employed for the encoding process. The image decoding apparatus 200 can correctly decode the encoded data generated by the image encoding apparatus 100 by performing a decoding process based on the information.

[Macro block]
The size of the macroblock may be 16 × 16 or less, but may be larger than 16 × 16.

  The present invention can be applied to macroblocks of any size as shown in FIG. For example, the present invention can be applied not only to a normal macroblock such as 16 × 16 pixels but also to an extended macroblock (extended macroblock) such as 32 × 32 pixels.

  In FIG. 25, in the upper part, from the left, a macro block composed of 32 × 32 pixels divided into blocks (partitions) of 32 × 32 pixels, 32 × 16 pixels, 16 × 32 pixels, and 16 × 16 pixels. Are shown in order. Further, in the middle stage, blocks from 16 × 16 pixels divided into blocks of 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, and 8 × 8 pixels are sequentially shown from the left. . Further, in the lower part, from the left, an 8 × 8 pixel block divided into 8 × 8 pixel, 8 × 4 pixel, 4 × 8 pixel, and 4 × 4 pixel blocks is sequentially shown.

  That is, the 32 × 32 pixel macroblock can be processed in the 32 × 32 pixel, 32 × 16 pixel, 16 × 32 pixel, and 16 × 16 pixel blocks shown in the upper part.

  The block of 16 × 16 pixels shown on the right side of the upper row is H.264. Similar to the H.264 / AVC format, processing in blocks of 16 × 16 pixels, 16 × 8 pixels, 8 × 16 pixels, and 8 × 8 pixels shown in the middle stage is possible.

  The 8 × 8 pixel block shown on the right side of the middle row is H.264. Similar to the H.264 / AVC format, processing in blocks of 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, and 4 × 4 pixels shown in the lower stage is possible.

  These blocks can be classified into the following three layers. That is, a block of 32 × 32 pixels, 32 × 16 pixels, and 16 × 32 pixels shown in the upper part of FIG. 25 is referred to as a first layer. The 16 × 16 pixel block shown on the right side of the upper stage and the 16 × 16 pixel, 16 × 8 pixel, and 8 × 16 pixel blocks shown in the middle stage are referred to as a second hierarchy. The 8 × 8 pixel block shown on the right side of the middle row and the 8 × 8 pixel, 8 × 4 pixel, 4 × 8 pixel, and 4 × 4 pixel blocks shown on the lower row are referred to as a third hierarchy.

  By adopting such a hierarchical structure, H. Larger blocks can be defined as a superset while maintaining compatibility with the H.264 / AVC format.

<4. Fourth Embodiment>
[Personal computer]
The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, a personal computer as shown in FIG. 26 may be configured.

  In FIG. 26, a CPU (Central Processing Unit) 601 of the personal computer 600 performs various processes according to a program stored in a ROM (Read Only Memory) 602 or a program loaded from a storage unit 613 into a RAM (Random Access Memory) 603. Execute the process. The RAM 603 also appropriately stores data necessary for the CPU 601 to execute various processes.

  The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output interface 610 is also connected to the bus 604.

  The input / output interface 610 includes an input unit 611 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display), an output unit 612 including a speaker, a hard disk, and the like. A communication unit 614 including a storage unit 613 and a modem is connected. The communication unit 614 performs communication processing via a network including the Internet.

  A drive 615 is connected to the input / output interface 610 as necessary, and a removable medium 621 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is loaded. It is installed in the storage unit 613 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 26, the recording medium is distributed to distribute the program to the user separately from the apparatus main body, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk ( It only consists of removable media 621 consisting of CD-ROM (compact disc-read only memory), DVD (including digital versatile disc), magneto-optical disc (including MD (mini disc)), or semiconductor memory. Rather, it is composed of a ROM 602 storing a program and a hard disk included in the storage unit 613, which is distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus composed of a plurality of devices (apparatuses).

  In addition, in the above description, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit). Of course, a configuration other than that described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  For example, the image encoding device 100 and the image decoding device 200 described above can be applied to any electronic device. Examples thereof will be described below.

<5. Fifth embodiment>
[Television receiver]
FIG. 27 is a block diagram illustrating a main configuration example of a television receiver using the image decoding device 200 to which the present invention has been applied.

  A television receiver 1000 shown in FIG. 27 includes a terrestrial tuner 1013, a video decoder 1015, a video signal processing circuit 1018, a graphic generation circuit 1019, a panel drive circuit 1020, and a display panel 1021.

  The terrestrial tuner 1013 receives a broadcast wave signal of analog terrestrial broadcasting via an antenna, demodulates it, acquires a video signal, and supplies it to the video decoder 1015. The video decoder 1015 performs a decoding process on the video signal supplied from the terrestrial tuner 1013 and supplies the obtained digital component signal to the video signal processing circuit 1018.

  The video signal processing circuit 1018 performs predetermined processing such as noise removal on the video data supplied from the video decoder 1015 and supplies the obtained video data to the graphic generation circuit 1019.

  The graphic generation circuit 1019 generates video data of a program to be displayed on the display panel 1021, image data by processing based on an application supplied via a network, and the generated video data and image data to the panel drive circuit 1020. Supply. The graphic generation circuit 1019 generates video data (graphics) for displaying a screen used by the user for selecting an item and superimposing it on the video data of the program. A process of supplying data to the panel drive circuit 1020 is also appropriately performed.

  The panel drive circuit 1020 drives the display panel 1021 based on the data supplied from the graphic generation circuit 1019 and causes the display panel 1021 to display a program video and the various screens described above.

  The display panel 1021 includes an LCD (Liquid Crystal Display) or the like, and displays a program video or the like according to control by the panel drive circuit 1020.

  The television receiver 1000 also includes an audio A / D (Analog / Digital) conversion circuit 1014, an audio signal processing circuit 1022, an echo cancellation / audio synthesis circuit 1023, an audio amplification circuit 1024, and a speaker 1025.

  The terrestrial tuner 1013 acquires not only the video signal but also the audio signal by demodulating the received broadcast wave signal. The terrestrial tuner 1013 supplies the acquired audio signal to the audio A / D conversion circuit 1014.

  The audio A / D conversion circuit 1014 performs A / D conversion processing on the audio signal supplied from the terrestrial tuner 1013 and supplies the obtained digital audio signal to the audio signal processing circuit 1022.

  The audio signal processing circuit 1022 performs predetermined processing such as noise removal on the audio data supplied from the audio A / D conversion circuit 1014, and supplies the obtained audio data to the echo cancellation / audio synthesis circuit 1023.

  The echo cancellation / voice synthesis circuit 1023 supplies the voice data supplied from the voice signal processing circuit 1022 to the voice amplification circuit 1024.

  The audio amplifying circuit 1024 performs D / A conversion processing and amplification processing on the audio data supplied from the echo cancellation / audio synthesizing circuit 1023, adjusts to a predetermined volume, and then outputs the audio from the speaker 1025.

  Furthermore, the television receiver 1000 also includes a digital tuner 1016 and an MPEG decoder 1017.

  The digital tuner 1016 receives a broadcast wave signal of a digital broadcast (terrestrial digital broadcast, BS (Broadcasting Satellite) / CS (Communications Satellite) digital broadcast) via an antenna, demodulates, and MPEG-TS (Moving Picture Experts Group). -Transport Stream) and supply it to the MPEG decoder 1017.

  The MPEG decoder 1017 cancels the scramble applied to the MPEG-TS supplied from the digital tuner 1016, and extracts a stream including program data to be played back (viewing target). The MPEG decoder 1017 decodes the audio packet constituting the extracted stream, supplies the obtained audio data to the audio signal processing circuit 1022, decodes the video packet constituting the stream, and converts the obtained video data into the video This is supplied to the signal processing circuit 1018. Also, the MPEG decoder 1017 supplies EPG (Electronic Program Guide) data extracted from the MPEG-TS to the CPU 1032 via a path (not shown).

  The television receiver 1000 uses the above-described image decoding device 200 as the MPEG decoder 1017 for decoding video packets in this way. Note that MPEG-TS transmitted from a broadcasting station or the like is encoded by the image encoding device 100.

  As in the case of the image decoding device 200, the MPEG decoder 1017 decodes the encoded data supplied from the image encoding device 100 in parallel for each entropy slice, and generates decoded image data. At that time, as in the case of the image decoding apparatus 200, the MPEG decoder 1017 divides the pre-encoding image corresponding to the encoded data to be decoded into a plurality (even number) of regions arranged vertically. Then, the MPEG decoder 1017 decodes the encoded data corresponding to the even-numbered regions from the top in the order from the top macroblock line to the bottom, as in the past, and left one macroblock to be processed as a peripheral macroblock. The macroblock and the macroblock one level above are referenced.

  On the other hand, for the encoded data corresponding to the odd-numbered area from the top, the MPEG decoder 1017 decodes in order from the bottom macroblock line to the top, contrary to the conventional case, and processes it as a peripheral macroblock. Reference is made to the macro block one left of the macro block and the macro block one level below.

  Further, when the decoding of the encoded data corresponding to the bottommost macroblock line in the odd-numbered area from the top is completed, the MPEG decoder 1017 encodes the code in the area one lower (even from the top). The decoding process for the digitized data is started.

  At this time, the MPEG decoder 1017 refers to the bottom macroblock line in the odd-numbered area from the top, and the top macroblock line in the area one lower (even from the top). The decoding process is performed on the encoded data corresponding to.

  Therefore, the MPEG decoder 1017 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

  The video data supplied from the MPEG decoder 1017 is subjected to predetermined processing in the video signal processing circuit 1018 as in the case of the video data supplied from the video decoder 1015, and the generated video data in the graphic generation circuit 1019. Are appropriately superimposed and supplied to the display panel 1021 via the panel drive circuit 1020, and the image is displayed.

  The audio data supplied from the MPEG decoder 1017 is subjected to predetermined processing in the audio signal processing circuit 1022 as in the case of the audio data supplied from the audio A / D conversion circuit 1014, and an echo cancellation / audio synthesis circuit 1023. Are supplied to the audio amplifier circuit 1024 through which D / A conversion processing and amplification processing are performed. As a result, sound adjusted to a predetermined volume is output from the speaker 1025.

  The television receiver 1000 also includes a microphone 1026 and an A / D conversion circuit 1027.

  The A / D conversion circuit 1027 receives a user's voice signal captured by a microphone 1026 provided in the television receiver 1000 for voice conversation, and performs A / D conversion processing on the received voice signal. The obtained digital audio data is supplied to the echo cancellation / audio synthesis circuit 1023.

  When the audio data of the user (user A) of the television receiver 1000 is supplied from the A / D conversion circuit 1027, the echo cancellation / audio synthesis circuit 1023 performs echo cancellation on the audio data of the user A. The voice data obtained by combining with other voice data is output from the speaker 1025 via the voice amplifier circuit 1024.

  The television receiver 1000 further includes an audio codec 1028, an internal bus 1029, an SDRAM (Synchronous Dynamic Random Access Memory) 1030, a flash memory 1031, a CPU 1032, a USB (Universal Serial Bus) I / F 1033, and a network I / F 1034. .

  The A / D conversion circuit 1027 receives a user's voice signal captured by a microphone 1026 provided in the television receiver 1000 for voice conversation, and performs A / D conversion processing on the received voice signal. The obtained digital audio data is supplied to the audio codec 1028.

  The audio codec 1028 converts the audio data supplied from the A / D conversion circuit 1027 into data of a predetermined format for transmission via the network, and supplies the data to the network I / F 1034 via the internal bus 1029.

  The network I / F 1034 is connected to the network via a cable attached to the network terminal 1035. For example, the network I / F 1034 transmits the audio data supplied from the audio codec 1028 to another device connected to the network. In addition, the network I / F 1034 receives, for example, audio data transmitted from another device connected via the network via the network terminal 1035, and receives the audio data via the internal bus 1029 to the audio codec 1028. Supply.

  The audio codec 1028 converts the audio data supplied from the network I / F 1034 into data of a predetermined format, and supplies it to the echo cancellation / audio synthesis circuit 1023.

  The echo cancellation / speech synthesis circuit 1023 performs echo cancellation on the speech data supplied from the speech codec 1028, and synthesizes speech data obtained by combining with other speech data via the speech amplification circuit 1024. And output from the speaker 1025.

  The SDRAM 1030 stores various data necessary for the CPU 1032 to perform processing.

  The flash memory 1031 stores a program executed by the CPU 1032. The program stored in the flash memory 1031 is read by the CPU 1032 at a predetermined timing such as when the television receiver 1000 is activated. The flash memory 1031 also stores EPG data acquired via digital broadcasting, data acquired from a predetermined server via a network, and the like.

  For example, the flash memory 1031 stores MPEG-TS including content data acquired from a predetermined server via a network under the control of the CPU 1032. The flash memory 1031 supplies the MPEG-TS to the MPEG decoder 1017 via the internal bus 1029, for example, under the control of the CPU 1032.

  The MPEG decoder 1017 processes the MPEG-TS as in the case of the MPEG-TS supplied from the digital tuner 1016. In this way, the television receiver 1000 receives content data including video and audio via the network, decodes it using the MPEG decoder 1017, displays the video, and outputs audio. Can do.

  The television receiver 1000 also includes a light receiving unit 1037 that receives an infrared signal transmitted from the remote controller 1051.

  The light receiving unit 1037 receives infrared light from the remote controller 1051 and outputs a control code representing the content of the user operation obtained by demodulation to the CPU 1032.

  The CPU 1032 executes a program stored in the flash memory 1031 and controls the overall operation of the television receiver 1000 according to a control code supplied from the light receiving unit 1037. The CPU 1032 and each part of the television receiver 1000 are connected via a path (not shown).

  The USB I / F 1033 transmits and receives data to and from a device external to the television receiver 1000 connected via a USB cable attached to the USB terminal 1036. The network I / F 1034 is connected to the network via a cable attached to the network terminal 1035, and transmits / receives data other than audio data to / from various devices connected to the network.

  By using the image decoding apparatus 200 as the MPEG decoder 1017, the television receiver 1000 can correctly decode encoded data encoded at high speed while suppressing a decrease in encoding efficiency. As a result, the television receiver 1000 can perform high-speed decoding of broadcast wave signals received via an antenna and content data obtained via a network, and realize real-time processing at a lower cost. Can do.

<6. Sixth Embodiment>
[Mobile phone]
FIG. 28 is a block diagram illustrating a main configuration example of a mobile phone using the image encoding device 100 and the image decoding device 200 to which the present invention is applied.

  A mobile phone 1100 shown in FIG. 28 has a main control unit 1150, a power supply circuit unit 1151, an operation input control unit 1152, an image encoder 1153, a camera I / F unit 1154, an LCD control, which are configured to control each unit in an integrated manner. Section 1155, image decoder 1156, demultiplexing section 1157, recording / reproducing section 1162, modulation / demodulation circuit section 1158, and audio codec 1159. These are connected to each other via a bus 1160.

  The mobile phone 1100 includes an operation key 1119, a CCD (Charge Coupled Devices) camera 1116, a liquid crystal display 1118, a storage unit 1123, a transmission / reception circuit unit 1163, an antenna 1114, a microphone (microphone) 1121, and a speaker 1117.

  When the end call and the power key are turned on by the user's operation, the power supply circuit unit 1151 starts up the mobile phone 1100 in an operable state by supplying power from the battery pack to each unit.

  The mobile phone 1100 transmits and receives voice signals, e-mails and image data, and images in various modes such as a voice call mode and a data communication mode based on the control of the main control unit 1150 including a CPU, a ROM, a RAM, and the like. Various operations such as shooting or data recording are performed.

  For example, in the voice call mode, the mobile phone 1100 converts the voice signal collected by the microphone (microphone) 1121 into digital voice data by the voice codec 1159, performs spectrum spread processing by the modulation / demodulation circuit unit 1158, and transmits and receives The unit 1163 performs digital / analog conversion processing and frequency conversion processing. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (voice signal) transmitted to the base station is supplied to the mobile phone of the other party via the public telephone line network.

  Further, for example, in the voice call mode, the cellular phone 1100 amplifies the received signal received by the antenna 1114 by the transmission / reception circuit unit 1163, further performs frequency conversion processing and analog-digital conversion processing, and performs spectrum despreading processing by the modulation / demodulation circuit unit 1158. Then, the audio codec 1159 converts it into an analog audio signal. The cellular phone 1100 outputs an analog audio signal obtained by the conversion from the speaker 1117.

  Further, for example, when transmitting an e-mail in the data communication mode, the mobile phone 1100 receives e-mail text data input by operating the operation key 1119 in the operation input control unit 1152. The cellular phone 1100 processes the text data in the main control unit 1150 and displays it on the liquid crystal display 1118 as an image via the LCD control unit 1155.

  In addition, the mobile phone 1100 generates e-mail data in the main control unit 1150 based on text data received by the operation input control unit 1152, user instructions, and the like. The cellular phone 1100 performs spread spectrum processing on the e-mail data by the modulation / demodulation circuit unit 1158 and digital / analog conversion processing and frequency conversion processing by the transmission / reception circuit unit 1163. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (e-mail) transmitted to the base station is supplied to a predetermined destination via a network and a mail server.

  Further, for example, when receiving an e-mail in the data communication mode, the mobile phone 1100 receives and amplifies the signal transmitted from the base station by the transmission / reception circuit unit 1163 via the antenna 1114, and further performs frequency conversion processing and Analog-digital conversion processing. The cellular phone 1100 performs spectrum despreading processing on the received signal by the modulation / demodulation circuit unit 1158 to restore the original e-mail data. The cellular phone 1100 displays the restored e-mail data on the liquid crystal display 1118 via the LCD control unit 1155.

  Note that the mobile phone 1100 can also record (store) the received e-mail data in the storage unit 1123 via the recording / playback unit 1162.

  The storage unit 1123 is an arbitrary rewritable storage medium. The storage unit 1123 may be, for example, a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, or a removable disk such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. It may be media. Of course, other than these may be used.

  Furthermore, for example, when transmitting image data in the data communication mode, the mobile phone 1100 generates image data with the CCD camera 1116 by imaging. The CCD camera 1116 has an optical device such as a lens and a diaphragm and a CCD as a photoelectric conversion element, images a subject, converts the intensity of received light into an electrical signal, and generates image data of the subject image. The CCD camera 1116 encodes the image data by the image encoder 1153 via the camera I / F unit 1154 and converts the encoded image data into encoded image data.

  The cellular phone 1100 uses the above-described image encoding device 100 as the image encoder 1153 that performs such processing. The image encoder 1153 divides the image to be encoded into a plurality of (even number) regions arranged in the vertical direction as in the case of the image encoding device 100. Then, the image encoder 1153 encodes the even-numbered regions from the top in the order from the top macroblock line to the bottom in the conventional manner, and sets the macroblock one left of the processing target macroblock as the peripheral macroblock. Refers to the next higher macroblock.

  On the other hand, in the odd-numbered area from the top, the image encoder 1153 performs coding in the order from the bottom macroblock line to the top, contrary to the conventional case, and sets 1 of the processing target macroblock as a peripheral macroblock. Refers to the left macroblock and the next lower macroblock.

  Further, the image encoder 1153 starts encoding processing for the next lower (even number from the top) area when encoding of the bottommost macroblock line in the odd numbered area from the top is completed. To do.

  At this time, the image encoder 1153 refers to the bottom macroblock line in the odd-numbered area from the top, and the top macroblock line in the area one level lower (even from the top). The encoding process for is performed.

  By performing such encoding, the image encoder 1153 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

  At the same time, the cellular phone 1100 converts the sound collected by the microphone (microphone) 1121 during imaging by the CCD camera 1116 from analog to digital at the audio codec 1159 and further encodes it.

  The cellular phone 1100 multiplexes the encoded image data supplied from the image encoder 1153 and the digital audio data supplied from the audio codec 1159 in a demultiplexing unit 1157 using a predetermined method. The cellular phone 1100 performs spread spectrum processing on the multiplexed data obtained as a result by the modulation / demodulation circuit unit 1158 and digital / analog conversion processing and frequency conversion processing by the transmission / reception circuit unit 1163. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. A transmission signal (image data) transmitted to the base station is supplied to a communication partner via a network or the like.

  When image data is not transmitted, the mobile phone 1100 can also display the image data generated by the CCD camera 1116 on the liquid crystal display 1118 via the LCD control unit 1155 without using the image encoder 1153.

  Further, for example, when receiving data of a moving image file linked to a simple homepage or the like in the data communication mode, the mobile phone 1100 transmits a signal transmitted from the base station to the transmission / reception circuit unit 1163 via the antenna 1114. Receive, amplify, and further perform frequency conversion processing and analog-digital conversion processing. The cellular phone 1100 restores the original multiplexed data by subjecting the received signal to spectrum despreading processing by the modulation / demodulation circuit unit 1158. In the cellular phone 1100, the demultiplexing unit 1157 separates the multiplexed data and divides it into encoded image data and audio data.

  The cellular phone 1100 generates reproduced moving image data by decoding the encoded image data in the image decoder 1156, and displays it on the liquid crystal display 1118 via the LCD control unit 1155. Thereby, for example, the moving image data included in the moving image file linked to the simple homepage is displayed on the liquid crystal display 1118.

  The cellular phone 1100 uses the above-described image decoding device 200 as the image decoder 1156 that performs such processing. That is, as in the case of the image decoding device 200, the image decoder 1156 decodes the encoded data supplied from the image encoding device 100 in parallel for each entropy slice, and generates decoded image data. At that time, as in the case of the image decoding apparatus 200, the image decoder 1156 divides the pre-encoding image corresponding to the encoded data to be decoded into a plurality of (even number) regions arranged vertically. Then, the image decoder 1156 decodes the encoded data corresponding to the even-numbered regions from the top in the order from the top macroblock line to the bottom as in the past, and left one macroblock to be processed as a peripheral macroblock. The macroblock and the macroblock one level above are referenced.

  On the other hand, for the encoded data corresponding to the odd-numbered region from the top, the image decoder 1156 decodes in order from the bottom macroblock line to the top, contrary to the conventional case, and is processed as a peripheral macroblock. Reference is made to the macro block one left of the macro block and the macro block one level below.

  Furthermore, when the decoding of the encoded data corresponding to the lowermost macroblock line in the odd-numbered area from the top is completed, the image decoder 1156 encodes the code of the area below it (even-numbered from the top). The decoding process for the digitized data is started.

  At this time, the image decoder 1156 refers to the bottom macroblock line in the odd-numbered area from the top, and the top macroblock line in the area one lower (even from the top). The decoding process is performed on the encoded data corresponding to.

  Therefore, the image decoder 1156 can realize high-speed image encoding while suppressing a decrease in encoding efficiency.

  At this time, the cellular phone 1100 simultaneously converts digital audio data into an analog audio signal in the audio codec 1159 and outputs the analog audio signal from the speaker 1117. Thereby, for example, audio data included in the moving image file linked to the simple homepage is reproduced.

  As in the case of e-mail, the mobile phone 1100 can record (store) the data linked to the received simplified home page in the storage unit 1123 via the recording / playback unit 1162. .

  Further, the mobile phone 1100 can analyze the two-dimensional code captured by the CCD camera 1116 and acquire information recorded in the two-dimensional code in the main control unit 1150.

  Further, the cellular phone 1100 can communicate with an external device by infrared rays at the infrared communication unit 1181.

  The cellular phone 1100 uses the image encoding device 100 as the image encoder 1153, thereby reducing the encoding efficiency by speeding up the image encoding when the image data generated by the CCD camera 1116 is encoded and transmitted, for example. Can be suppressed. As a result, the mobile phone 1100 can realize real-time processing at a lower cost.

  In addition, the mobile phone 1100 uses the image decoding apparatus 200 as the image decoder 1156, for example, data of a moving image file linked to a simple homepage or the like (encoded at high speed while suppressing reduction in encoding efficiency). Encoded data) can be correctly decoded, and real-time processing can be realized at a lower cost.

  In the above description, the mobile phone 1100 is described as using the CCD camera 1116. However, instead of the CCD camera 1116, an image sensor (CMOS image sensor) using a CMOS (Complementary Metal Oxide Semiconductor) is used. May be. Also in this case, the mobile phone 1100 can capture an image of a subject and generate image data of the image of the subject, as in the case where the CCD camera 1116 is used.

  In the above description, the mobile phone 1100 has been described. For example, PDA (Personal Digital Assistants), a smartphone, an UMPC (Ultra Mobile Personal Computer), a netbook, a notebook personal computer, and the like. As long as it is a device having a communication function, the image encoding device 100 and the image decoding device 200 can be applied to any device as in the case of the mobile phone 1100.

<7. Seventh Embodiment>
[Hard Disk Recorder]
FIG. 29 is a block diagram illustrating a main configuration example of a hard disk recorder using the image encoding device 100 and the image decoding device 200 to which the present invention is applied.

  A hard disk recorder (HDD recorder) 1200 shown in FIG. 29 receives audio data and video data of a broadcast program included in a broadcast wave signal (television signal) transmitted from a satellite or a ground antenna received by a tuner. This is an apparatus for storing in a built-in hard disk and providing the stored data to the user at a timing according to the user's instruction.

  The hard disk recorder 1200 can extract, for example, audio data and video data from broadcast wave signals, appropriately decode them, and store them in a built-in hard disk. The hard disk recorder 1200 can also acquire audio data and video data from other devices via a network, for example, decode them as appropriate, and store them in a built-in hard disk.

  Further, the hard disk recorder 1200, for example, decodes audio data and video data recorded on the built-in hard disk, supplies them to the monitor 1260, displays the image on the screen of the monitor 1260, and displays the sound from the speaker of the monitor 1260. Can be output. Further, the hard disk recorder 1200 decodes audio data and video data extracted from a broadcast wave signal acquired via a tuner, or audio data and video data acquired from another device via a network, for example. The image can be supplied to the monitor 1260, the image can be displayed on the screen of the monitor 1260, and the sound can be output from the speaker of the monitor 1260.

  Of course, other operations are possible.

  As illustrated in FIG. 29, the hard disk recorder 1200 includes a receiving unit 1221, a demodulating unit 1222, a demultiplexer 1223, an audio decoder 1224, a video decoder 1225, and a recorder control unit 1226. The hard disk recorder 1200 further includes an EPG data memory 1227, a program memory 1228, a work memory 1229, a display converter 1230, an OSD (On Screen Display) control unit 1231, a display control unit 1232, a recording / playback unit 1233, a D / A converter 1234, And a communication unit 1235.

  In addition, the display converter 1230 includes a video encoder 1241. The recording / playback unit 1233 includes an encoder 1251 and a decoder 1252.

  The receiving unit 1221 receives an infrared signal from a remote controller (not shown), converts it into an electrical signal, and outputs it to the recorder control unit 1226. The recorder control unit 1226 is constituted by, for example, a microprocessor and executes various processes according to a program stored in the program memory 1228. At this time, the recorder control unit 1226 uses the work memory 1229 as necessary.

  The communication unit 1235 is connected to a network and performs communication processing with other devices via the network. For example, the communication unit 1235 is controlled by the recorder control unit 1226, communicates with a tuner (not shown), and mainly outputs a channel selection control signal to the tuner.

  The demodulator 1222 demodulates the signal supplied from the tuner and outputs the demodulated signal to the demultiplexer 1223. The demultiplexer 1223 separates the data supplied from the demodulation unit 1222 into audio data, video data, and EPG data, and outputs them to the audio decoder 1224, the video decoder 1225, or the recorder control unit 1226, respectively.

  The audio decoder 1224 decodes the input audio data and outputs it to the recording / playback unit 1233. The video decoder 1225 decodes the input video data and outputs it to the display converter 1230. The recorder control unit 1226 supplies the input EPG data to the EPG data memory 1227 for storage.

  The display converter 1230 encodes the video data supplied from the video decoder 1225 or the recorder control unit 1226 into, for example, NTSC (National Television Standards Committee) video data by the video encoder 1241 and outputs the encoded video data to the recording / reproducing unit 1233. The display converter 1230 converts the screen size of the video data supplied from the video decoder 1225 or the recorder control unit 1226 into a size corresponding to the size of the monitor 1260, and converts the video data to NTSC video data by the video encoder 1241. Then, it is converted into an analog signal and output to the display control unit 1232.

  The display control unit 1232 superimposes the OSD signal output from the OSD (On Screen Display) control unit 1231 on the video signal input from the display converter 1230 under the control of the recorder control unit 1226, and displays it on the monitor 1260 display. Output and display.

  The monitor 1260 is also supplied with the audio data output from the audio decoder 1224 after being converted into an analog signal by the D / A converter 1234. The monitor 1260 outputs this audio signal from a built-in speaker.

  The recording / playback unit 1233 includes a hard disk as a storage medium for recording video data, audio data, and the like.

  For example, the recording / reproducing unit 1233 encodes the audio data supplied from the audio decoder 1224 by the encoder 1251. The recording / playback unit 1233 encodes the video data supplied from the video encoder 1241 of the display converter 1230 by the encoder 1251. The recording / playback unit 1233 combines the encoded data of the audio data and the encoded data of the video data by a multiplexer. The recording / playback unit 1233 amplifies the synthesized data by channel coding, and writes the data to the hard disk via the recording head.

  The recording / playback unit 1233 plays back the data recorded on the hard disk via the playback head, amplifies it, and separates it into audio data and video data by a demultiplexer. The recording / playback unit 1233 uses the decoder 1252 to decode the audio data and the video data. The recording / playback unit 1233 performs D / A conversion on the decoded audio data and outputs it to the speaker of the monitor 1260. In addition, the recording / playback unit 1233 performs D / A conversion on the decoded video data and outputs it to the display of the monitor 1260.

  The recorder control unit 1226 reads the latest EPG data from the EPG data memory 1227 based on the user instruction indicated by the infrared signal from the remote controller received via the receiving unit 1221, and supplies it to the OSD control unit 1231. To do. The OSD control unit 1231 generates image data corresponding to the input EPG data, and outputs the image data to the display control unit 1232. The display control unit 1232 outputs the video data input from the OSD control unit 1231 to the display of the monitor 1260 for display. As a result, an EPG (electronic program guide) is displayed on the display of the monitor 1260.

  Also, the hard disk recorder 1200 can acquire various data such as video data, audio data, or EPG data supplied from another device via a network such as the Internet.

  The communication unit 1235 is controlled by the recorder control unit 1226, acquires encoded data such as video data, audio data, and EPG data transmitted from another device via the network, and supplies the encoded data to the recorder control unit 1226. To do. For example, the recorder control unit 1226 supplies the encoded data of the acquired video data and audio data to the recording / playback unit 1233 and stores it in the hard disk. At this time, the recorder control unit 1226 and the recording / playback unit 1233 may perform processing such as re-encoding as necessary.

  Also, the recorder control unit 1226 decodes the acquired encoded data of video data and audio data, and supplies the obtained video data to the display converter 1230. Similar to the video data supplied from the video decoder 1225, the display converter 1230 processes the video data supplied from the recorder control unit 1226, supplies the processed video data to the monitor 1260 via the display control unit 1232, and displays the image. .

  In accordance with the image display, the recorder control unit 1226 may supply the decoded audio data to the monitor 1260 via the D / A converter 1234 and output the sound from the speaker.

  Further, the recorder control unit 1226 decodes the encoded data of the acquired EPG data, and supplies the decoded EPG data to the EPG data memory 1227.

  The hard disk recorder 1200 as described above uses the image decoding device 200 as a decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control unit 1226. That is, the decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control unit 1226, in the same way as the case of the image decoding device 200, parallelizes the encoded data supplied from the image encoding device 100 for each entropy slice. To generate decoded image data. At that time, as in the case of the image decoding device 200, the video decoder 1225, the decoder 1252, and the decoder built in the recorder control unit 1226 move the image before encoding corresponding to the encoded data to be decoded up and down. Divide into multiple (even) areas. The video decoder 1225, the decoder 1252, and the decoder incorporated in the recorder control unit 1226 decode the encoded data corresponding to the even-numbered regions from the top in the order from the top macroblock line to the bottom as before. Then, the macroblock one left of the processing target macroblock and the macroblock one level higher are referred to as the peripheral macroblock.

  On the other hand, for the encoded data corresponding to the odd-numbered area from the top, the decoder built in the video decoder 1225, the decoder 1252, and the recorder control unit 1226, contrary to the conventional case, Decoding is performed in order from the top to the bottom, and the macroblock one left of the processing target macroblock and the macroblock one below are referred to as neighboring macroblocks.

  Furthermore, when the decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control unit 1226 finishes decoding the encoded data corresponding to the bottommost macroblock line in the odd-numbered region from the top, The decoding process for the encoded data in the next lower area (even number from the top) is started.

  At this time, the decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control unit 1226 refers to the lowermost macroblock line in the odd-numbered area from the top (one above) The decoding process is performed on the encoded data corresponding to the top macroblock line in the (even-numbered) area.

  Therefore, the video decoder 1225, the decoder 1252, and the decoder built in the recorder control unit 1226 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

  Therefore, the hard disk recorder 1200, for example, video data received by the tuner or communication unit 1235 (encoded data encoded at high speed while suppressing reduction in encoding efficiency) or video data reproduced by the recording / reproducing unit 1233. (Encoded data encoded at high speed while suppressing a reduction in encoding efficiency) can be correctly decoded, and real-time processing can be realized at a lower cost.

  The hard disk recorder 1200 uses the image encoding device 100 as the encoder 1251. Therefore, the encoder 1251 divides the image to be encoded into a plurality of (even number) regions arranged in the vertical direction, as in the case of the image encoding device 100. The encoder 1251 then encodes the even-numbered regions from the top in the order from the top macroblock line to the bottom in the conventional manner, and the macroblock one left of the processing target macroblock as the surrounding macroblock and 1 Refers to the next macroblock.

  On the other hand, in the odd-numbered region from the top, the encoder 1251 performs coding in the order from the bottom macroblock line to the top, contrary to the conventional case, and sets one of the processing target macroblocks as a peripheral macroblock. Refer to the left macroblock and the next lower macroblock.

  Furthermore, the encoder 1251 starts the encoding process for the next lower (even number from the top) area when the encoding of the bottommost macroblock line in the odd numbered area from the top is completed. .

  At this time, the encoder 1251 refers to the bottom macroblock line in the odd-numbered area from the top, and relates to the top macroblock line in the area one lower (even from the top). The encoding process is performed.

  By performing such encoding, the encoder 1251 can realize high-speed image encoding while suppressing a decrease in encoding efficiency.

  Therefore, for example, the hard disk recorder 1200 can suppress a reduction in encoding efficiency due to high-speed image encoding when generating encoded data to be recorded on the hard disk. As a result, the mobile phone 1100 can realize real-time processing at a lower cost.

  In the above description, the hard disk recorder 1200 for recording video data and audio data on the hard disk has been described. Of course, any recording medium may be used. For example, even in a recorder to which a recording medium other than a hard disk such as a flash memory, an optical disk, or a video tape is applied, the image encoding device 100 and the image decoding device 200 are applied as in the case of the hard disk recorder 1200 described above. Can do.

<8. Eighth Embodiment>
[camera]
FIG. 30 is a block diagram illustrating a main configuration example of a camera using the image encoding device 100 and the image decoding device 200 to which the present invention is applied.

  A camera 1300 shown in FIG. 30 images a subject and displays an image of the subject on the LCD 1316 or records it on the recording medium 1333 as image data.

  The lens block 1311 causes light (that is, an image of the subject) to enter the CCD / CMOS 1312. The CCD / CMOS 1312 is an image sensor using CCD or CMOS, converts the intensity of received light into an electric signal, and supplies it to the camera signal processing unit 1313.

  The camera signal processing unit 1313 converts the electrical signal supplied from the CCD / CMOS 1312 into Y, Cr, and Cb color difference signals, and supplies them to the image signal processing unit 1314. The image signal processing unit 1314 performs predetermined image processing on the image signal supplied from the camera signal processing unit 1313 or encodes the image signal with the encoder 1341 under the control of the controller 1321. The image signal processing unit 1314 supplies encoded data generated by encoding the image signal to the decoder 1315. Further, the image signal processing unit 1314 acquires display data generated in the on-screen display (OSD) 1320 and supplies it to the decoder 1315.

  In the above processing, the camera signal processing unit 1313 appropriately uses a DRAM (Dynamic Random Access Memory) 1318 connected via the bus 1317, and image data or a code obtained by encoding the image data as necessary. The digitized data or the like is held in the DRAM 1318.

  The decoder 1315 decodes the encoded data supplied from the image signal processing unit 1314 and supplies the obtained image data (decoded image data) to the LCD 1316. In addition, the decoder 1315 supplies the display data supplied from the image signal processing unit 1314 to the LCD 1316. The LCD 1316 appropriately synthesizes the image of the decoded image data supplied from the decoder 1315 and the image of the display data, and displays the synthesized image.

  Under the control of the controller 1321, the on-screen display 1320 outputs display data such as menu screens and icons composed of symbols, characters, or graphics to the image signal processing unit 1314 via the bus 1317.

  The controller 1321 executes various processes based on a signal indicating the content instructed by the user using the operation unit 1322, and also via the bus 1317, an image signal processing unit 1314, a DRAM 1318, an external interface 1319, an on-screen display. 1320, media drive 1323, and the like are controlled. The FLASH ROM 1324 stores programs and data necessary for the controller 1321 to execute various processes.

  For example, the controller 1321 can encode the image data stored in the DRAM 1318 or decode the encoded data stored in the DRAM 1318 instead of the image signal processing unit 1314 or the decoder 1315. At this time, the controller 1321 may be configured to perform encoding / decoding processing by a method similar to the encoding / decoding method of the image signal processing unit 1314 or the decoder 1315, or the image signal processing unit 1314 or the decoder 1315 is compatible. The encoding / decoding process may be performed by a method that is not performed.

  For example, when the start of image printing is instructed from the operation unit 1322, the controller 1321 reads out image data from the DRAM 1318 and supplies it to the printer 1334 connected to the external interface 1319 via the bus 1317. Let it print.

  Further, for example, when image recording is instructed from the operation unit 1322, the controller 1321 reads the encoded data from the DRAM 1318 and supplies it to the recording medium 1333 mounted on the media drive 1323 via the bus 1317. Remember.

  The recording medium 1333 is an arbitrary readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Of course, the recording medium 1333 may be of any kind as a removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.

  Further, the media drive 1323 and the recording medium 1333 may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or SSD (Solid State Drive).

  The external interface 1319 is composed of, for example, a USB input / output terminal, and is connected to the printer 1334 when printing an image. In addition, a drive 1331 is connected to the external interface 1319 as necessary, and a removable medium 1332 such as a magnetic disk, an optical disk, or a magneto-optical disk is appropriately mounted, and a computer program read from them is loaded as necessary. Installed in the FLASH ROM 1324.

  Furthermore, the external interface 1319 has a network interface connected to a predetermined network such as a LAN or the Internet. For example, the controller 1321 can read the encoded data from the DRAM 1318 in accordance with an instruction from the operation unit 1322 and supply the encoded data to the other device connected via the network from the external interface 1319. In addition, the controller 1321 acquires encoded data and image data supplied from another device via the network via the external interface 1319, holds the data in the DRAM 1318, or supplies it to the image signal processing unit 1314. Can be.

  The camera 1300 as described above uses the image decoding device 200 as the decoder 1315. That is, as in the case of the image decoding device 200, the decoder 1315 decodes the encoded data supplied from the image encoding device 100 in parallel for each entropy slice, and generates decoded image data. At that time, as in the case of the image decoding device 200, the decoder 1315 divides the pre-encoding image corresponding to the encoded data to be decoded into a plurality (even number) of regions arranged vertically. Then, the decoder 1315 decodes the encoded data corresponding to the even-numbered regions from the top in the order from the top macroblock line to the bottom as in the past, and the left side of the processing target macroblock is left as the surrounding macroblock. Refers to the macroblock and the macroblock one level above.

  On the other hand, for the encoded data corresponding to the odd-numbered area from the top, the decoder 1315 decodes in order from the bottom macroblock line to the top, contrary to the conventional case, and performs processing as the surrounding macroblock. Reference is made to the macroblock one block to the left and the macroblock one block below.

  Furthermore, when the decoding of the encoded data corresponding to the bottommost macroblock line in the odd-numbered area from the top is completed, the decoder 1315 encodes the area below (even-numbered from the top). The decryption process for data is started.

  At this time, the decoder 1315 refers to the bottom macroblock line in the odd-numbered area from the top, and refers to the top macroblock in the area one lower (even from the top). A decoding process is performed on the encoded data corresponding to the line.

  Therefore, the decoder 1315 can realize high-speed image encoding while suppressing reduction in encoding efficiency.

  Therefore, the camera 1300, for example, encodes image data generated in the CCD / CMOS 1312, encoded data of video data read from the DRAM 1318 or the recording medium 1333, and encoded efficiency of encoded data of video data acquired via the network. The real-time processing can be realized at a lower cost.

  The camera 1300 uses the image encoding device 100 as the encoder 1341. The encoder 1341 divides the image to be encoded into a plurality (even number) of regions that are vertically arranged, as in the case of the image encoding device 100. Then, the encoder 1341 encodes the even-numbered regions from the top in the order from the top macroblock line to the bottom in the conventional manner, and the macroblock one left of the processing target macroblock as the surrounding macroblock and 1 Refers to the next macroblock.

  On the other hand, for the odd-numbered area from the top, the encoder 1341 performs coding in the order from the bottom macroblock line to the top, contrary to the conventional case, and sets one of the processing target macroblocks as a peripheral macroblock. Refer to the left macroblock and the next lower macroblock.

  Furthermore, the encoder 1341 starts the encoding process for the next lower (even number from the top) area when the encoding of the bottom macroblock line in the odd numbered area from the top is completed. .

  At this time, the encoder 1341 refers to the bottom macroblock line in the odd-numbered area from the top, and relates to the top macroblock line in the area one lower (even from the top). The encoding process is performed.

  By performing such encoding, the encoder 1341 can achieve high-speed image encoding while suppressing a decrease in encoding efficiency.

  Therefore, the camera 1300 uses the image encoding device 100 as the encoder 1341, for example, to speed up image encoding of encoded data to be recorded in the DRAM 1318 or the recording medium 1333 or encoded data to be provided to other devices. It is possible to suppress a reduction in encoding efficiency due to. As a result, the camera 1300 can realize real-time processing at a lower cost.

  Note that the decoding method of the image decoding device 200 may be applied to the decoding process performed by the controller 1321. Similarly, the encoding method of the image encoding device 100 may be applied to the encoding process performed by the controller 1321.

  The image data captured by the camera 1300 may be a moving image or a still image.

  Of course, the image encoding device 100 and the image decoding device 200 can also be applied to devices and systems other than the devices described above.

  100 image encoding device, 106 lossless encoding unit, 151 control unit, 152 storage unit, 153 slice 1 processing unit, 154 slice 2 processing unit, 200 image decoding device, 202 lossless decoding unit, 251 demultiplexer, 252 slice 1 processing Section, 253 slice 2 processing section, 301 context processing section, 501 control section, 503 slice 3 processing section, 504 slice 4 processing section, 551 demultiplexer, 552 slice 3 processing section, 554 slice 4 processing section

One aspect of the present technology is that when decoding encoded data in which image data is encoded, an upper block line that is a block line adjacent to an upper portion of a target block line to be decoded is positioned at the head of the target block line. Context updated when decoding the upper block, which is a block located in a decoding order later than the upper head block adjacent to the upper part of the target head block located, and updated when decoding the upper block An image including a decoding unit that initializes a context used when decoding the target head block using identification data that identifies whether the probability table is used as an initial value, and arithmetically decodes the target head block It is a processing device.

The decoding unit may arithmetically decode the target head block using a probability table updated when the upper block is decoded as an initial value.

The identification data is included in the encoded data, and the decoding unit can acquire the identification data from the encoded data .

In one aspect of the present technology, when decoding encoded data in which image data is encoded, an upper block line that is an upper block line adjacent to an upper portion of a target block line to be decoded is an area of the target block line. The context updated when the upper block, which is a block located in the decoding order later than the upper head block adjacent to the top of the target head block located at the top, is decoded, and when the upper block is decoded. An image processing method for initializing a context used when decoding the target head block using identification data for identifying whether to use the updated probability table as an initial value, and arithmetically decoding the target head block It is.

The target head block can be arithmetically decoded using the probability table updated when the upper block is decoded as an initial value.

The identification data can be acquired from the encoded data including the identification data .

In one aspect of the present technology, when the encoded data obtained by encoding the image data is further decoded by the computer, the target is detected in the upper block line which is a block line adjacent to the upper part of the target block line to be decoded. The context updated when decoding the upper block, which is a block located in a decoding order later than the upper head block adjacent to the top of the target head block located at the top of the block line, and the upper block The context data used when decoding the target head block is initialized using the identification data for identifying whether the updated probability table is used as an initial value, and the target head block is arithmetically decoded. It is a program for functioning as a decoding unit .

In one aspect of the present technology, when the encoded data in which the image data is encoded is decoded by the computer, the target block line is an upper block line that is adjacent to the upper part of the target block line to be decoded. The context updated when decoding the upper block, which is a block located in a decoding order later than the upper head block adjacent to the top of the target head block located at the top of the block line, and the upper block The context data used when decoding the target head block is initialized using the identification data for identifying whether the updated probability table is used as an initial value, and the target head block is arithmetically decoded. It is a computer-readable recording medium which records the program for functioning as a decoding part .

In one aspect of the present technology, when decoding encoded data in which image data is encoded, the top of the target block line in the upper block line that is adjacent to the upper part of the target block line to be decoded Context updated when decoding the upper block, which is a block located in the decoding order after the upper head block adjacent to the upper part of the target head block located at, and updated when decoding the upper block The identification data for identifying whether the probability table is used as an initial value is used, the context used when decoding the target head block is initialized, and the target head block is arithmetically decoded.

Claims (10)

An encoding unit that encodes a block of image data using a context;
It is a block located at a different location from the upper head block adjacent to the upper part of the target head block located at the head of the target block line in the upper block line that is a block line adjacent to the top of the target block line to be encoded. An image processing apparatus comprising: a control unit that controls the encoding unit so as to initialize a context used when encoding the target head block using a context updated when the upper block is encoded . The control unit sets, as an initial value, a context updated when the upper block located at a position later in the encoding order than the upper head block is encoded with respect to the upper block line, the target head block The image processing apparatus according to claim 1, wherein the encoding unit is controlled so as to encode the image. 3. The image processing according to claim 2, wherein the control unit controls the encoding unit to encode the target head block using, as an initial value, a probability table updated when the upper block is encoded. apparatus. The image processing device according to claim 3, wherein the encoding unit arithmetically encodes the target block line using a context. Encode a block of image data using context,
It is a block located at a different location from the upper head block adjacent to the upper part of the target head block located at the head of the target block line in the upper block line that is a block line adjacent to the top of the target block line to be encoded. An image processing method for controlling to initialize a context used when encoding the target head block using a context updated when an upper block is encoded. The target head block is encoded with the context updated when the upper block located in the encoding order after the upper head block is encoded with respect to the upper block line as an initial value. The image processing method according to claim 5. The image processing method according to claim 6, wherein control is performed so that the target head block is encoded with a probability table updated when the upper block is encoded as an initial value. The image processing method according to claim 7, wherein the target block line is arithmetically encoded using a context. On the computer,
An encoding unit that encodes a block of image data using a context;
It is a block located at a different location from the upper head block adjacent to the upper part of the target head block located at the head of the target block line in the upper block line that is a block line adjacent to the top of the target block line to be encoded. To function as a control unit that controls the encoding unit so as to initialize the context used when encoding the target head block using the context updated when the upper block is encoded Program. On the computer,
An encoding unit that encodes a block of image data using a context;
It is a block located at a different location from the upper head block adjacent to the upper part of the target head block located at the head of the target block line in the upper block line that is a block line adjacent to the top of the target block line to be encoded. To function as a control unit that controls the encoding unit so as to initialize the context used when encoding the target head block using the context updated when the upper block is encoded A computer-readable recording medium on which the program is recorded.

Images