JP2015002541A - Image decoding device and image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the number of times of access to a reference memory without installing an intermediate memory.SOLUTION: An image decoding device comprises: a first buffer for storing compression image data; a second buffer for storing a decoded picture; decoding means which refers to the picture stored in the second buffer to decode the compression image data stored in the first buffer and stores the decoded picture in the second buffer; count means for counting the number of MB meeting a specific condition within the decoded picture; and control means for controlling operation of the entire device. The control means determines a picture type of image data being stored in the first buffer, makes a boundary determination to determine a boundary to be defined as an alternative picture without performing decoding processing in accordance with a result of the determination and a count result of the count means and determines the alternative picture without decoding a predetermined number of pictures after the picture determined as the boundary.

Description

  The present invention relates to an image decoding apparatus and an image decoding method, and more particularly to a technique suitable for use in decoding compressed image data compressed by interframe predictive encoding.

  Since the resolution of an image tends to increase and the data size of the image to be handled increases accordingly, a compression encoding technique with a higher compression rate is desired. At present, H.264 / MPEG4-AVC (hereinafter referred to as H.264) is the mainstream of moving image compression coding systems. In general, in compression encoding of moving images, a high compression rate is realized by reducing redundancy in the time direction and space direction of image data. For example, when encoding, a predicted image is generated from an already encoded picture, and the difference from the predicted image is encoded, thereby reducing the size of data to be generated.

At the time of decoding, a decoding process is performed by generating a prediction image from a picture referred to when generating a prediction image and adding the prediction image at the time of encoding.
At the time of encoding, a picture that does not refer to a predicted picture is called an I picture, a picture that references one predicted picture is called a P picture, and a picture that references two predicted pictures is called a B picture. In addition, reference to pictures at the time of encoding and decoding is performed in units of macroblocks (hereinafter referred to as MB). In H.264, at the time of encoding, it is possible to select one of a plurality of MB sizes to be referred to, such as 16 × 16 or 8 × 8.

  In consumer cameras that use compression encoding technology such as H.264, save the generated video to media, set the target code amount for playback on the player, and protect this target code amount Is encoded.

  For example, when video with motion is encoded, the predicted image generated from the encoded picture is likely to deviate, and the data size of the difference to be encoded becomes large, so the MB size is encoded with 16 × 16. By doing so, it tries to reduce the code amount.

  On the other hand, when a video such as a still image with little motion is encoded, the predicted image is easy to hit, so the difference is close to zero. When there is no difference, the data size called Skip MB is replaced with a small MB, so that the code amount is increased by finely encoding with 8 × 8 MB. Therefore, the compressed and encoded data of a motionless video tends to have Skip MB and 8 × 8 MB continuous.

  When decoding a P-picture or B-picture where 8 × 8 MB is frequently generated, access to a memory (hereinafter referred to as a reference memory) in which the decoded picture is stored occurs more frequently than 16 × 16 MB. A large amount of memory bus rate is consumed. In particular, since the B picture refers to two predicted images, the load on the reference memory increases and the overhead increases compared to the P picture, so the load increases as the resolution of the image data to be decoded increases. To go.

  In a consumer camera, the predicted image at the time of encoding is likely to be lost due to camera shake of the photographer or sensor noise, and it is rare to generate compressed encoded data in which 8 × 8 MB and Skip MB described above frequently occur. For this reason, the decoding device mounted on the camera may be designed with limitations so that decoding can be performed only when the rate of occurrence of 8 × 8 MB generated in one picture is below a certain level from the viewpoint of cost. is there.

  Such a decoding device is guaranteed to decode the self-recorded image data. However, for example, when trying to decode image data frequently generated by 8 × 8 MB generated by an editing device or the like, The reference memory may be accessed frequently and cannot be reproduced accurately.

As a method for reducing the number of accesses to the reference memory, a compression encoding device and a decoding device are known in which an intermediate memory is installed and a reference picture is read after being read into the intermediate memory.
For example, in Patent Document 1, when compression encoding is performed with reference to an encoded picture, the reference picture is read from the reference memory to the intermediate memory. If there are multiple pictures that refer to the picture in the read intermediate memory, the reference picture is referenced in parallel while switching the picture in MB units, and compression coding is performed, thereby reducing the number of accesses to the reference memory. Let

  In addition, by decoding the image data compressed and encoded by the above-described method using an intermediate memory, the same effect as that at the time of compression encoding can be obtained at the time of decoding.

JP 2009-290387 A

However, in the configuration of the conventional technique described above, it is necessary to install an intermediate memory in order to reduce access to the reference memory. In addition, since the decoding apparatus described in Patent Document 1 assumes that image data compressed and encoded by the encoding apparatus described in Patent Document 1 is input, it is compressed and encoded by another method. When decoding the received image data, the above-described effects cannot be obtained, so that it is not general purpose.
The present invention has been made in view of the above-described problems, and an object thereof is to reduce the number of accesses to a reference memory without installing an intermediate memory.

  An image decoding apparatus according to the present invention comprises: a first buffer that stores the compressed image data; and a decoding apparatus that decodes compressed image data that has been compression-encoded using inter-frame predictive encoding. A second buffer for storing the decoded picture and a picture stored in the second buffer, the compressed image data stored in the first buffer is decoded, and the decoded picture is Decoding means for storing in the second buffer; counting means for counting the number of encoded blocks satisfying a specific condition in the picture decoded by the decoding means; and control means for controlling the operation of the entire apparatus The control means includes a picture type determination means for determining a picture type of the image data stored in the first buffer, and the picture type determination. A boundary determination unit that determines a boundary to be an alternative picture instead of performing the decoding process from the determination result of the unit and the count result of the counting unit, and is determined to be a boundary by the boundary determination unit After the picture, a predetermined number of pictures are not decoded and replaced with alternative pictures.

  According to the present invention, in the process of decoding image data, even when a plurality of image data frequently accessed to the reference memory are continuously input, the access to the reference memory can be performed without an additional circuit such as a memory. The number of times can be reduced. Thereby, the failure of the system can be avoided at a low cost.

1 is a block diagram illustrating an exemplary configuration of an image decoding device according to an embodiment of the present invention. It is a flowchart explaining the procedure in which an image decoding apparatus decodes image data. It is a flowchart explaining the procedure which judges the reference | standard of a decoding process skip in 1st and 2nd embodiment. It is a flowchart explaining the procedure which judges the decoding process skip in 1st Embodiment. It is explanatory drawing of the judgment process of a decoding process skip in 1st Embodiment. It is a flowchart explaining the procedure which sets a counter in 2nd Embodiment. It is a flowchart explaining the procedure which judges the decoding process skip in 2nd Embodiment. It is explanatory drawing of the determination process of a decoding process skip in 2nd Embodiment. It is a flowchart explaining the process sequence which judges the reference | standard of a decoding process skip in 3rd Embodiment. It is a flowchart explaining the process sequence which judges the decoding process skip in 3rd Embodiment. It is explanatory drawing of the determination process of a decoding process skip in 3rd Embodiment. It is a block diagram which shows the structural example of the recording / reproducing apparatus in 4th Embodiment. It is a block diagram explaining the detailed structural example of an encoding process part. It is a flowchart which shows 4th Embodiment and demonstrates an example of the procedure of an encoding process. It is a block diagram which shows the structural example of the recording / reproducing apparatus in 5th Embodiment. It is a flowchart which shows 5th Embodiment and demonstrates an example of the procedure of an encoding process. It is a figure explaining the arrangement | positioning method of the screen periphery of S1603 of the flowchart of FIG.

Hereinafter, preferred embodiments of an image decoding apparatus of the present invention will be described with reference to the drawings.
(First embodiment)
In the present embodiment, in the decoding process of image data compressed and encoded by H.264, an example in which the decoding process is skipped while a B picture continues after a picture satisfying a certain condition and an alternative picture is displayed instead. Will be described. In addition, the system of the image decoding apparatus in each embodiment is not limited to H.264, and other similar encoding systems can be similarly applied.

FIG. 1 is a block diagram illustrating an example of an image decoding apparatus adapted to the present embodiment.
The image decoding apparatus 100 in FIG. 1 includes a CPB 101, a decoding unit 102, a DPB 103, a picture information storage unit 104, a control unit 105, and a bus 106, and performs compression encoding using inter-frame prediction encoding. The compressed image data is decoded.

CPB (Coded Picture Buffers) 101 is a first buffer provided for storing compression-encoded data input from an image input unit (not shown), and stores image data at a constant bit rate. Used to decrypt.
The decoding unit 102 performs decoding processing based on the control of the control unit 105, and has a decoding function and a counting function of the number of encoded blocks (specific MB). Specifically, it has the following three functions (a) to (c).

(A) A function for decoding input data compressed and encoded by the H.264 method.
(B) A function for calculating the ratio of SkipMB in a picture during decoding.
(C) A function for calculating a ratio of 8 × 8 MB in a picture at the time of decoding.

  SkipMB is an MB used to reduce the amount of data when there is no difference between the reference portion and the encoding target portion when compression encoding is performed with inter-frame predictive encoding. The decoding unit 102 writes the above-described information in units of pictures obtained by the decoding process into the picture information storage unit 104, and writes the decoded picture into a DPB (Decoded Picture Buffers) 103.

  The DPB 103 is a second buffer provided for storing decoded pictures. The DPB 103 rearranges decoded pictures and the image data compressed by the decoding unit 102 by inter-frame prediction encoding. Used to store a picture to be referenced when decoding.

  The picture information storage unit 104 manages information such as Skip MB ratio, 8 × 8 MB ratio, picture type, and picture size in one picture obtained by decoding processing in units of pictures.

  The control unit 105 controls the image decoding apparatus 100, that is, controls the operation of the entire apparatus, and includes a picture type determination unit and a boundary determination unit. Specifically, the control unit 105 determines the picture type of the image data stored in the CPB 101 and writes the determination result in the picture information storage unit 104. Also, based on the picture information stored in the picture information storage unit 104, a criterion determination process for skipping the decoding process and a decoding process skip determination process are performed. Details of these two processes will be described later.

  The bus 106 exchanges data between the blocks of the CPB 101 to the control unit 105. Hereinafter, description of processing of the bus 106 when data is transferred between the blocks is omitted.

Next, the overall processing of the image decoding apparatus 100 will be described with reference to FIG. The processing of the flowcharts described below with reference to FIGS. 2 to 4 is realized by developing a program stored in the ROM provided in the control unit 105 in the RAM and executing it by the CPU.
FIG. 2 is a flowchart of a process in which the image decoding apparatus decodes one piece of image data.
In step S201, the control unit 105 determines whether a picture that is a reference for skipping the decoding process is set.

If a picture serving as a reference for skipping decoding processing is set, decoding processing skip determination processing is performed in S202. If it has not been set, a criterion judgment process for skipping the decoding process is performed in S203.
When the process of S202 or 203 ends, the decoded data is output in S204, and the process ends.
The above processing is repeated until all input pictures are displayed.

Next, the decoding determination skip criterion determination process of S203 will be described with reference to the flowchart of FIG.
FIG. 3 is a flowchart illustrating a procedure for determining a criterion for skipping the decoding process.
Immediately before the image data stored in the CPB 101 is decoded, the control unit 105 determines the picture type from the slice header of the image data and writes it in the picture information storage unit 104.

In step S <b> 301, the control unit 105 instructs the decoding unit 102 to decode the image data input to the CPB 101.
The decoding unit 102 performs image data decoding processing. When decoding interframe predictive-encoded image data, for example, a B picture or a P picture, the decoding unit 102 refers to the decoded picture stored in the DPB 103 as appropriate, and performs a decoding process. When compression encoding is performed by intra-frame prediction, the decoding process is performed without reference.

  In parallel with the decoding process, the decoding unit 102 counts the number of SkipMB and 8 × 8 MB of image data. Then, after decoding the image data, calculate what percentage SkipMB and 8 × 8 MB occupy the total number of MBs, write to the picture information storage unit 104, and write the decoded picture to the DPB 103.

  In step S <b> 302, the control unit 105 refers to the picture type of the image data decoded by the decoding unit 102 from the picture information storage unit 104 and determines whether the decoded image data is a B picture. If the decoded image data is a B picture, the process proceeds to S303. If it is not a B picture, the process ends.

  In S303, the control unit 105 refers to the rate of SkipMB of the image data decoded by the decoding unit 102 from the picture information storage unit 104, and determines whether the rate of SkipMB is greater than or equal to a preset threshold value α. Judging. For example, when the threshold value is 70%, if SkipMB is 70% or more with respect to the total number of MBs of the image data, the process proceeds to S304. If it is less than 70%, the process is terminated.

  In S304, the control unit 105 refers to the 8 × 8 MB ratio of the image data decoded by the decoding unit 102 from the picture information storage unit 104, and the 8 × 8 MB ratio is equal to or higher than a preset threshold value β. Judge whether it is less than β. For example, when the threshold value is 70%, if 8 × 8 MB is 70% or more with respect to the number of MBs of the entire image data, the process proceeds to S305. If it is less than 70%, the process is terminated.

  In step S305, the control unit 105 sets the image data that is a B picture and the ratio of Skip MB to 8 × 8 MB exceeds 70% as a reference picture for skipping decoding processing, and ends the reference determination processing.

Next, the process of determining whether to skip the decoding process in S202 will be described with reference to FIGS. FIG. 4 is a flowchart of the decoding process skip determination process. FIG. 5 is an explanatory diagram of the decoding process skip determination process.
Assume that the arrangement of the image data input to the CPB 101 is the input order of FIG. 5 and that B1 is input after B0 is determined as the reference picture in the process of S203. B indicates the picture type, and 0 indicates the display order.

In step S <b> 401, the control unit 105 determines whether the picture type is a B picture from the slice header of B <b> 1 that is image data input to the CPB 101. If the image data is a B picture, the process proceeds to S402. If it is not a B picture, the process proceeds to S404.
Since B1 is a B picture, the process proceeds to S402.

In S402, the control unit 105 skips the decoding process without instructing the decoding unit 102 to perform the decoding process of B1.
In step S403, the control unit 105 sets a substitute picture to be displayed instead of the B1 decoding result.

For example, as alternative picture candidates,
(A) B1 reference picture.
(B) The closest picture in the display order of B1.
Etc. are conceivable.

In the present embodiment, the closest picture in the display order of the target picture is set as a substitute picture (see FIG. 5). Therefore, the decoded B0 picture stored in the DPB 103 is set as a substitute picture, and in S204, the control unit 105 causes the substitute picture B0 picture to be output instead of the decoding result of B1.
When P5, which is the next input image data, is input to the CPB 101, in S401, the control unit 105 determines that it is not a B picture, and proceeds to the processing of S404.

In step S404, since the B picture is interrupted, the control unit 105 resets a picture that is a reference for skipping the decoding process.
In step S405, the control unit 105 instructs the decoding unit 102 to decode the image data P5 input to the CPB 101, and the decoding unit 102 performs the decoding process. In S <b> 204, the control unit 105 outputs the next I2 picture in the display order stored in the DPB 103.
Thereafter, the processing from S201 to S204 is repeated for the input image data.

FIG. 5 shows the processing result described above. The displayed picture sequence is “B0, B0, I2, B3, B4, P5”.
As described above, in this embodiment, as long as the B picture continues after the picture that is the reference for skipping the decoding process, the alternative picture is displayed instead of performing the decoding process of the target picture. As a result, even when a plurality of image data frequently accessed to the reference memory are continuously input, the system failure can be avoided by reducing the access to the reference memory.

(Second Embodiment)
In the present embodiment, in the decoding process of image data compressed and encoded by H.264, if a B picture continues after a picture satisfying a certain condition and the frame rate is high, the decoding process skip period is lengthened. . In the case of a low frame rate, an example in which the skip period of the decoding process is shortened will be described.

The configuration of the image decoding apparatus according to the present embodiment is the same as that of FIG. 1 that is the configuration of the first embodiment. The overall process and the decoding process skip criterion determination process are also the same as those in the flowcharts of FIGS. 2 and 3 used in the first embodiment, and a description thereof will be omitted.
6 and 7 is realized by developing a program stored in a ROM provided in the control unit 105 in a RAM and executing the program by the CPU.

With reference to FIG. 6, a description will be given of a counter setting process that determines how many decoding processes to skip when B pictures continue after the reference picture.
FIG. 6 is a flowchart for explaining the procedure of the counter setting process.
In step S601, the control unit 105 determines whether the frame rate of the image data input to the CPB 101 is greater than or equal to a preset threshold value γ or less than γ. For example, when the threshold is 60 fps, if the frame rate of the input image data is 60 fps or more, the process proceeds to S602. If it is less than 60 fps, the process proceeds to S603.

In S602 and S603, the control unit 105 sets values X and Y (X> Y) for the variable i.
The variable i is a variable indicating how many B pictures the decoding process is skipped from the picture that is the reference for the decoding process skip. For example, the control unit 105 stores a predetermined value such as X = 2 and Y = 1 in the variable i, and ends the counter setting process.
The counter setting process is performed on the next image data input to the CPB 101 to be decoded.

Next, processing for determining whether to skip decoding processing will be described with reference to FIGS.
FIG. 7 is a flowchart for explaining the procedure of the decoding process skip determination process. Since S401, S402, S403, S404, and S405 are the same as those in the flowchart of FIG.

  FIG. 8 is an explanatory diagram of the decoding process skip determination process. In the present embodiment, it is assumed that image data is input to the CPB 101 in the input order of FIG. The processing will be described below on the assumption that image data is input from B1 after B0 is set as a reference for skipping decoding processing.

In S401, it is determined whether or not the picture type of B1 to be decoded next is a B picture. If it is a B picture, the process proceeds to S701. If it is not a B picture, the process proceeds to S703.
Since B1 is the same picture type as the B picture, in step S701, the control unit 105 determines whether or not the value of the variable i is 0. If the variable i is not 0, the process proceeds to S702. If the variable i is 0, the process proceeds to S703.
Since variable i is set to 2 in the counter setting process, the process proceeds to S702.

In S702, the control unit 105 decrements the variable i, skips the decoding process in S402 and S403, and sets a substitute picture B0.
The above processing is repeated until the B picture is interrupted or the variable i becomes 0.
When the input image data is B3, in S702, the control unit 105 determines that the variable i is 0, and proceeds to S703.

In S703, the control unit 105 sets the variable i to the value set in the counter setting process, performs the processes in S404 and S405, and ends the process.
Thereafter, the processing from S201 to S204 is repeated for the input image data.

FIG. 8 shows the processing result described above. The displayed picture sequence is “B0, B0, B0, B3, I4”.
As described above, in the present embodiment, when the input frame rate is high, it is determined that adjacent images are similar, and the decoding process skip period is set to be long. If the frame rate is low, it is determined that the difference between adjacent images is large, and the skip period of the decoding process is set short.

  Subsequent pictures are displayed instead of the decoding process of the target picture without performing the decoding process of the target picture during the set skip period after the picture that becomes the reference for the decoding process skip. As a result, even when a plurality of image data frequently accessed to the reference memory is input, it is possible to avoid a system failure by reducing the access to the reference memory.

(Third embodiment)
In the present embodiment, in the decoding process of compression-encoded image data, a B picture continues after a picture satisfying a certain condition, and a picture having a size greater than or equal to a predetermined value appears from a picture satisfying the condition. An example in which decoding processing is skipped will be described.
The configuration of this embodiment is the same as that of FIG. 1 which is the configuration of the first embodiment. Moreover, since the whole process is the same as the flowchart of FIG. 2 used in the first embodiment, the description thereof is omitted.

A picture size threshold setting process used when determining whether to skip the decoding process will be described with reference to FIG.
FIG. 9 is a process for determining a decoding process skip criterion. The processing of S301, S302, S303, S304, and S305 is the same as the processing of the flowchart of FIG.

  When image data that is a B picture, the rate of SkipMB is greater than or equal to the threshold value α, and the rate of 8 × 8 MB is greater than or equal to the threshold value β is decoded in S305, the control unit 105 performs the above-described conditions. A picture that satisfies the above is set as a reference picture.

  In step S901, the control unit 105 acquires the data size of the reference picture before decoding from the picture information storage unit 104, and sets a threshold for the picture size from the acquired data size. The threshold value set here is used to determine whether to skip the decoding process.

As a threshold setting method, for example, if the decoding process is skipped even for a picture that is somewhat different from the reference picture, set a large threshold and skip only pictures with little difference. Set the threshold value smaller.
In the present embodiment, since a picture with little difference from the reference picture is a target of skip processing,
"Threshold Z = size of reference picture N + 3k bytes"
The threshold value Z is set as follows. After setting the threshold value, the process is terminated.

Next, processing for determining whether to skip decoding processing will be described with reference to FIGS. 10 and 11.
FIG. 10 is a flowchart of the decoding process skip determination process. Since S401, S402, S403, S404, and S405 are the same as those in the flowchart of FIG.

  FIG. 11 is an explanatory diagram of the decoding process skip determination process of the present embodiment. In the present embodiment, it is assumed that image data is input to the CPB 101 in the input order of FIG. When the picture size of B0 is N, the size of B1 is N + 1k bytes, and the size of B2 is N + 4k bytes.

The processing will be described below assuming that image data is input from B1 after B0 is set as a reference picture for decoding processing skipping.
In S401, it is determined whether or not the picture type of B1 to be decoded next is a B picture. If it is a B picture, the process proceeds to S1001. If it is not a B picture, the process proceeds to S404.

  Since B1 is a B picture, in S1001, the control unit 105 obtains the B1 picture size from the picture information storage unit 104, and determines whether the B1 picture size exceeds the threshold value Z set in S901. To do. If it is less than the threshold value Z, the process proceeds to S402 and the decoding process is skipped. If it is equal to or greater than the threshold value Z, the process advances to step S404 to reset a picture that serves as a reference for skipping the decoding process and perform a decoding process.

  Since the picture size of B1 is N + 1k bytes, it becomes less than the threshold value Z. In S402 and S403, the nearest B0 picture in the display order is set as the alternative picture, and the decoding process is skipped.

In the next input B2, since the size of B2 is N + 4k bytes in S1001, it is determined that there is a motion with respect to the reference picture B0 because it is equal to or greater than the threshold value Z. In S404 and S405, the reference picture Perform reset and B2 decryption. Thereafter, the processing from S201 to S204 is repeated for the input image data.
The processing results described above are shown in FIG. The displayed picture sequence is “B0, B0, B2, B3, I4”.

  As described above, in this embodiment, a picture size threshold value is set in advance, and if the target picture size is equal to or larger than the threshold value, it is determined that the picture size is significantly different from the reference picture, and normal decoding processing is performed. If it is a B picture and the picture size is less than the threshold value, it is determined that the motion is small, and instead of decoding the target picture, an alternative picture is displayed instead. As a result, even when a plurality of image data frequently accessed to the reference memory is input, it is possible to avoid a system failure by reducing the access to the reference memory.

(Fourth embodiment)
Hereinafter, preferred embodiments of an image encoding device of the present invention will be described in detail with reference to the accompanying drawings. FIG. 12 is a block diagram showing a configuration example of a recording / reproducing apparatus to which the image coding apparatus of the present invention can be applied.

Hereinafter, the configuration of the entire system according to the fourth embodiment of the present invention will be described with reference to FIG.
In FIG. 12, reference numeral 1201 denotes a photographing lens including a focus lens, 1202 denotes an iris having a diaphragm function, and 1203 denotes an imaging unit including a CCD or a CMOS element sensor that converts an optical image into an electric signal. An A / D converter 1204 converts an analog signal into a digital signal. The A / D converter 1204 is used to convert an analog signal output from the imaging unit 1203 into a digital signal.

  An image processing unit 1205 performs resize processing and color conversion processing such as camera signal processing, predetermined pixel interpolation, and reduction on the data from the A / D converter 1204. The image processing unit 1205 performs predetermined calculation processing using the captured image data, and the system control unit 1208 performs exposure control and distance measurement control based on the obtained calculation result.

  Thereby, AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-emission) processing of the TTL (through-the-lens) method are performed. The image processing unit 1205 further performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

Output data from the A / D converter 1204 is written in the memory 1207 via the camera signal processing in the image processing unit 1205. The memory 1207 stores image data obtained by the imaging unit 1203 and converted into digital data by the A / D converter 1204 and the image processing unit 1205. The memory 1207 has a storage capacity sufficient to store a predetermined number of still images, a moving image and sound for a predetermined time.
Reference numeral 1206 denotes an encoding processing unit that encodes original image data input from the memory 1207 into a predetermined format. In the present embodiment, a plurality of blocked blocks are collected and encoded for each macroblock.

  The nonvolatile memory 1211 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. The nonvolatile memory 1211 stores constants for operating the system control unit 1208, programs, and the like. Here, the program is a program for executing various flowcharts described later in the present embodiment.

  A system control unit 1208 controls the entire recording / reproducing apparatus 1200. By executing the program recorded in the non-volatile memory 1211 described above, each process of the present embodiment to be described later is realized. A system memory 1210 uses a RAM. In the system memory 1210, constants and variables for operation of the system control unit 1208, a program read from the nonvolatile memory 1211, and the like are expanded. The operation unit 1209 is an operation unit for inputting various operation instructions to the system control unit 1208.

  The system control unit 1208 controls an operation of a series of imaging processes from reading a signal from the imaging unit 1203 to writing image data to the recording medium 1222 in response to a shooting start request from the operation unit 1209. Further, the system control unit 1208 controls each unit of the recording / reproducing apparatus 1200 in accordance with input information from the operation unit 1209. Reference numeral 1221 denotes an interface with a recording medium 1222 such as a memory card or a hard disk. The recording medium 1222 is a recording medium such as a memory card, and includes a semiconductor memory or a magnetic disk.

FIG. 13 is a diagram illustrating details of the encoding processing unit 1206.
The original image data 1300 is data output by the image processing unit 1205. The original image data 1300 is once held in the memory 1207 described above. A memory 1207 is a common memory used in the entire system. The data output to the memory 1207 is referred to as original image data 1301.

The inter-frame prediction unit 1305 includes a motion detection unit 1305 (a) and a motion compensation unit 1305 (b), and performs inter-frame prediction processing based on the input original image data 1301 and local decoded data 1302 described later. The inter-frame prediction image data is generated.
An intra-frame / inter-frame determination unit 1304 performs intra-frame prediction processing using the input original image data 1301. Then, by comparing with the inter-frame prediction image data output from the inter-frame prediction unit 1305, it is determined whether either intra-frame prediction or inter-frame prediction is the final macroblock.

  The intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305 determine the block size in the macro block according to the characteristics of the input original image data. For example, when the image has a complicated pattern, the image quality is improved by using 8 × 8 macroblocks. Regardless of the characteristics of the input original image data, the system control unit 1208 can determine the block size in the macroblock by controlling the intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305. .

  The block counting unit 1306 counts macroblocks having a specific block size from the macroblocks determined by the intraframe / interframe determination unit 1304 and the interframe prediction unit 1305 and notifies the system control unit 1208 of the macroblocks. In this embodiment, 8 × 8 macroblocks are counted.

  When the macroblock type selected by the intra-frame / inter-frame determination unit 1304 is inter-frame prediction, the predicted macro-block image generated by the motion compensation unit 1305 (b) is the original image of the current frame by the difference unit 1303. Difference processing is performed with respect to the data 1301. Then, it is input to the orthogonal transform unit 1307 as a difference macroblock image. At this time, the predicted macroblock image is simultaneously output to an adder (not shown) in the local decoding unit 1308 via the 1311 path.

  The orthogonal transform unit 1307 performs orthogonal transform on the difference macroblock image output from the subtractor 1303 to obtain transform coefficients, and then quantizes the macroblock by applying a predetermined quantization scale to the transform coefficients. In this embodiment, DCT (Discrete Cosin Transfer) is used for orthogonal transform, but the present invention is not limited to this.

  The entropy encoding unit 1309 performs entropy encoding on the quantized data output from the orthogonal transform unit 1307. Furthermore, the mode information for intra-frame prediction obtained by the intra-frame / inter-frame determination unit 1304 and the motion information for inter-frame prediction obtained by the inter-frame prediction unit 1305 are also entropy-coded and output to the memory 1207. .

  The local decoding unit 1308 performs inverse quantization and inverse orthogonal exchange on the quantized transform coefficient output from the orthogonal transform unit 1307, generates local decoded image data, and outputs the local decoded image data to the memory 1207. The data output to the memory 1207 is referred to as local decoded data 1302. The code amount control unit 1310 adjusts the code amount so as to maintain the buffer model with respect to the determined target code amount.

Next, the operation of the inter-frame prediction unit 1305 will be described.
The inter-frame prediction unit 1305 includes a motion detection unit 1305 (a) and a motion compensation unit (b). The motion detection unit 1305 (a) evaluates the similarity between images from the input original image data 1301 and local decoded data 1302. The similarity is determined based on the result of sequential block matching while shifting the reference position within the search range, and the motion vector (motion information) of the processing target block is determined. Furthermore, the motion compensation unit 1305 (b) generates a predicted macroblock image extracted from the reference frame using the motion vector determined by the motion detection process, and outputs the predicted macroblock image as interframe predicted image data.

  FIG. 14 is a flowchart illustrating a control method for encoding according to the number of macroblocks having a specific block size in the fourth embodiment. Each process in this flowchart is realized by the system control unit 1208 developing a program stored in the nonvolatile memory 1211 in the system memory 1210 and executing the program.

  In step S1401, the system control unit 1208 reads the count notified from the block count unit 1306 from the system memory 1210, and checks whether the number of 8 × 8 macroblocks exceeds a predetermined threshold within one frame. When it exceeds, it progresses to S1402, and when it does not exceed, it progresses to S1403.

In step S1402, the system control unit 1208 controls the intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305 to perform intra-frame prediction processing and inter-frame prediction processing using 16 × 16 macroblocks in step S1404 described later. To do. Here, the 16 × 16 macroblock has a second block size and is larger than the 8 × 8 macroblock which is the first block size.
In S1403, the system control unit 1208 controls the intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305 so as to determine the block size in the macroblock according to the original image data in S1404 described later.

In step S1404, the system control unit 1208 controls the intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305 to generate intra-frame prediction data or inter-frame prediction data for each pixel.
In step S1405, the system control unit 1208 controls the block counting unit 1306 to count the number of macroblocks having a block size of 8 × 8 pixels among the macroblocks generated in step S1404.

In step S1406, the system control unit 1208 determines whether encoding of all pixels in the frame has been completed. If YES, the process proceeds to S1408, and the encoding of the frame is completed. If NO, the process proceeds to S37.
In step S <b> 1407, the system control unit 1208 controls the entropy encoding unit 1309 to entropy encode the quantized data output from the orthogonal transform unit 1307 and output the encoded data to the memory 1207.

  As described above, according to the fourth embodiment, when the number of 8 × 8 macroblocks exceeds a threshold in one frame, the remaining block of pixel data has the second block size or the second macro. Encode in block mode (intraframe reference macroblock). By adjusting the block size in the macroblock to be encoded at the time of encoding so that system failure does not occur at the time of decoding, it is possible to prevent the stop of the image due to an increase in memory access when decoding the image data.

Further, the macroblock counted in S1405 has a block size of 8 × 8 pixels, but the block size is not limited.
Furthermore, although the block size of the macroblock to be encoded is set to 16 × 16 pixels in S1402, the block size is not limited.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. In the fourth embodiment, when the number of macroblocks having a block size specified in one frame exceeds a threshold, the remaining pixels are encoded with a size larger than the block size. In the present embodiment, a macroblock arrangement method for encoding the next frame when the number of macroblocks having a designated block size exceeds a threshold will be described.

FIG. 15 is a block diagram showing a configuration example of a recording / reproducing apparatus 1200 to which the present invention can be applied. Since other than 1501 is the same as that of the fourth embodiment, the description is omitted here.
Reference numeral 1501 denotes a macroblock arrangement unit that determines the arrangement of macroblocks having a block size designated at the time of encoding.

  FIG. 16 is a flowchart for explaining a control method for controlling the arrangement of macroblocks at the time of encoding in the fifth embodiment. Each process in this flowchart is realized by the system control unit 1208 developing a program stored in the nonvolatile memory 1211 in the system memory 1210 and executing the program.

  In step S <b> 1601, the system control unit 1208 confirms whether the number of 8 × 8 macroblocks exceeds the threshold in the previous frame encoding from the count result held by the block counting unit 1306 in the system memory 1210. If exceeded, the process proceeds to S1602. If not, the process advances to step S1611 to execute the process of the flowchart described above with reference to FIG.

In step S1602, the system control unit 1208 determines whether the frame is a reference frame such as a B picture or a P picture. In the case of a non-reference frame such as an I picture, the process advances to step S1611 to execute the process of the flowchart in FIG.
In step S1603, the system control unit 1208 controls the macroblock placement unit 1501 to determine whether the macroblock is the periphery of the screen. If it is the periphery of the screen, the process proceeds to S1604. If it is not the periphery of the screen, the process proceeds to S1605. Details of this processing will be described later with reference to FIG.

In S1604, the system control unit 1208 uses a 16 × 16 macroblock having a second block size larger than the first block size of 8 × 8 macroblock in S1606 described later. Then, the intra-frame / inter-frame determination unit 1304 or the inter-frame prediction unit 1305 is controlled to perform intra-frame prediction processing or inter-frame prediction processing.
In step S1605, the system control unit 1208 controls the intra-frame / inter-frame determination unit 1304 or the inter-frame prediction unit 1305 to determine the block size and type in the macro block according to the original image data in step S1606 described later.

In step S1606, the system control unit 1208 controls the intra-frame / inter-frame determination unit 1304 and the inter-frame prediction unit 1305 to generate intra-frame prediction data or inter-frame prediction data for each pixel.
In S1607, the system control unit 1208 controls the block counting unit 1306 to count 8 × 8 macroblocks among the macroblocks generated in S1606.

In step S1608, the system control unit 1208 determines whether encoding of all pixels in the frame has been completed. If YES, the process advances to S1610 to complete the encoding of the frame. If NO, the process proceeds to S1609.
In step S <b> 1609, the system control unit 1208 controls the entropy encoding unit 1309 to entropy encode the quantized data output from the orthogonal transform unit 1307 and output the encoded data to the memory 1207.

FIG. 17 is a diagram for explaining the arrangement method around the screen in S1603 described above.
Original image data input as a frame to be encoded is encoded for each macroblock composed of a plurality of pixel groups obtained by dividing the frame into rectangular regions as indicated by reference numeral 1700 in FIG. Here, reference numeral 1710 denotes one macroblock.

The system control unit 1208 sets the number of 16 × 16 macroblocks (α) counted in the previous frame as the total number of macroblocks arranged in the periphery of the screen in this frame.
There are four types of parameters for the number of macroblocks used in the periphery of the screen: vertical vertical 1701 and 1702 and horizontal horizontal 1703 and 1704 in FIG. The macroblock arranging unit 1501 arranges 16 × 16 macroblocks so that the total number of 1701 to 1704 is within α described above.

  As described above, according to the fifth embodiment, macroblocks having a large block size are arranged in the periphery on the screen during encoding. Since the central portion of the screen is easily noticed by humans, image quality degradation is conspicuous if restricted to only 16 × 16 macroblocks in the center of the screen. Therefore, by arranging 16 × 16 macroblocks around the screen where the degree of attention of the person is reduced, it is possible to make the image quality degradation inconspicuous when decoding the encoding target frame.

  In the present embodiment, the block size determined in S1601 is 8 × 8 pixels, but the block size is not limited. Further, although the block size of the macro block measured in S1607 is 8 × 8 pixels, the block size is not limited. Furthermore, in S1604, the block size of the macroblock to be encoded is set to 16 × 16 pixels, but the block size is not limited.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

101 CPB
102 Decoding unit 103 DPB
104 Picture storage unit 105 Control unit 106 Bus

Claims (13)

In an image decoding apparatus for decoding compressed image data that has been compression-encoded using inter-frame prediction encoding,
A first buffer for storing the compressed image data;
A second buffer for storing decoded pictures;
Decoding means for decoding the compressed image data stored in the first buffer with reference to the picture stored in the second buffer and storing the decoded picture in the second buffer; ,
Counting means for counting the number of encoded blocks satisfying a specific condition in the picture decoded by the decoding means;
Control means for controlling the operation of the entire apparatus,
The control means includes
Picture type determination means for determining a picture type of image data stored in the first buffer;
Boundary determination means for determining a boundary as an alternative picture instead of performing decoding processing from the determination result of the picture type determination means and the count result of the counting means;
An image decoding apparatus characterized in that, after a picture determined to be a boundary by the boundary determination means, a predetermined number of pictures are not decoded and are used as substitute pictures.   The image decoding apparatus according to claim 1, wherein the predetermined number of images is until the same picture type continues.   2. The image decoding apparatus according to claim 1, wherein the control unit increases the predetermined number when the frame rate is high, and decreases the predetermined number when the frame rate is low.   2. The image decoding apparatus according to claim 1, wherein the predetermined number is set until a picture having a size larger than a predetermined value with respect to a size of the picture serving as the boundary appears.   The image decoding apparatus according to claim 1, wherein the substitute picture is selected as a picture closest to the display order of the target picture.   The image decoding apparatus according to claim 1, wherein the substitute picture is selected from pictures that the target picture refers to when decoding. Image input means;
Blocking means for collecting and blocking a plurality of pixels input by the image input means;
Encoding means for collecting a plurality of blocks blocked by the blocking means and encoding each block;
Block counting means for counting a first block size used when the encoding means encodes;
Holding means for holding the pixel data input by the image input means and the image data encoded by the encoding means in a memory;
Control means for controlling the operation of the entire apparatus,
The control means, when the value counted by the block counting means within one frame exceeds a predetermined value,
The remaining pixel data in the encoding target frame is encoded in a second block size larger than the first block size or in a second macroblock mode.   The image coding apparatus according to claim 7, wherein the second macroblock mode is an intra-frame reference macroblock. Macroblock arrangement means for determining arrangement of macroblocks on the screen when encoding using the second block size or the second macroblock mode;
When the count obtained by the block counting unit exceeds a predetermined value within one frame, the control unit controls the macroblock arranging unit to encode the second block size macro. 8. The image encoding apparatus according to claim 7, wherein an arrangement for encoding in a block or second macroblock mode is determined.   The macroblock arrangement means arranges a macroblock to be encoded with the second block size or a macroblock to be encoded with the second macroblock mode at the edge of the screen of the encoding target frame. Item 10. The image encoding device according to Item 9. In an image decoding method for decoding compressed image data that has been compression-encoded using inter-frame predictive encoding,
A storage step of storing the compressed image data in a first buffer;
Storing the decoded picture in a second buffer;
A decoding step of decoding the compressed image data stored in the first buffer with reference to the picture stored in the second buffer, and storing the decoded picture in the second buffer; ,
A counting step of counting the number of encoded blocks satisfying a specific condition in the picture decoded by the decoding step;
A control process for controlling the operation of the entire apparatus,
The control step includes
A picture type determination step of determining a picture type of image data stored in the first buffer;
From the determination result of the picture type determination step and the count result of the counting step, there is a boundary determination step of determining a boundary as an alternative picture instead of performing a decoding process,
An image decoding method characterized in that after a picture determined to be a boundary by the boundary determination step, a predetermined number of pictures are not decoded and are used as substitute pictures. In a program that causes a computer to execute an image decoding method for decoding compressed image data that has been compression-encoded using inter-frame predictive encoding,
A storage step of storing the compressed image data in a first buffer;
Storing the decoded picture in a second buffer;
A decoding step of decoding the compressed image data stored in the first buffer with reference to the picture stored in the second buffer, and storing the decoded picture in the second buffer; ,
A counting step of counting the number of encoded blocks satisfying a specific condition in the picture decoded by the decoding step;
Causing the computer to execute a control process for controlling the operation of the entire apparatus,
The control step includes
A picture type determination step of determining a picture type of image data stored in the first buffer;
From the determination result of the picture type determination step and the count result of the counting step, perform a boundary determination step of determining a boundary as an alternative picture instead of performing a decoding process,
A program for causing a computer to execute a substitute picture without decoding a predetermined number of pictures after a picture determined to be a boundary in the boundary determination step.   A computer-readable storage medium storing the program according to claim 12.

Images