JP2006246351A - Image coding unit and image decoding unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image coding unit having an improved coding efficiency without requiring coding of block information, by modifying a block of processing unit according to an image characteristic. <P>SOLUTION: An image coding unit includes an image input section 101 for inputting an original image; a down-sampling section 105 for obtaining a down-sampled image as a basic image of the original image; a difference processing section 102 for obtaining a difference image between the original image and the basic image; a block information acquisition section 108 for dividing the basic image into a plurality of blocks of different size based on edge information included in the basic image, and for obtaining block information indicating a state of the division; and an orthogonal transformation section 103 and an entropy coding section 104 for encoding the difference image, based on the plurality of blocks divided into the state indicated by the block information as a processing unit. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image encoding device and an image decoding device, and more particularly to an image encoding device and an image decoding device that can improve encoding efficiency by performing encoding with blocks suitable for image characteristics.

  In recent years, with the increase in the number of terrorism and crimes, video surveillance systems capable of connecting tens to hundreds of surveillance cameras are rapidly spreading in public facilities and the like. In addition, because the video from surveillance cameras is used as evidence images of crimes, the quality of still images, especially the resolution, is emphasized, and there is an effect of increasing the number of camera image sensors. Resolution is progressing.

  Along with the higher resolution of surveillance cameras, surveillance systems that connect dozens of surveillance cameras can efficiently compress high-resolution video in order to record long-term video with limited recording capacity. There is a demand for a video encoding method that can be realized.

  As a video encoding technique, for example, an encoding system using DCT (Discrete Cosine Transform) focusing on spatial redundancy of an image such as JPEG (Joint Picture Experts Group) system or MPEG (Moving Picture Experts Group) system is generally used. Is. In this MPEG encoding method, encoding by DCT conversion is performed using 8 × 8 pixels or 4 × 4 pixels as one processing block, and the block size is fixed.

  In the DCT processing, the DCT coefficient obtained by performing the DCT processing on a region where the pixel value in the image has little change (for example, the background portion) is biased to the low frequency region, so the DCT processing was performed using a large block. It is possible to improve the encoding efficiency. On the other hand, for regions where the pixel values in the image change a lot (for example, edge portions), the DCT coefficient obtained by DCT processing is biased toward the high frequency region, and pseudo-contour noise is generated when encoding is performed by removing high frequencies. Therefore, it is not suitable to use a large block size. Therefore, it is effective to increase the coding efficiency that the DCT block size is not fixed, but that the DCT block size is variable according to the image characteristics.

  In order to increase the coding efficiency by changing the DCT block size in this way, conventionally, as shown in Patent Document 1, for example, the DCT block size is determined using the pixel change value of the input image, and the DCT 2. Description of the Related Art An image coding apparatus that improves coding efficiency by coding a block size with a tree structure is known. FIG. 12 is a diagram illustrating a configuration of an image encoding device described in Patent Document 1. In FIG. 12, an image input unit 1200 inputs an image to be encoded. The block size assignment unit 1201 calculates a pixel change value of the input image, assigns a small DCT block size to an area where the pixel change value is large, and outputs the calculated block size to the DCT unit 1202. The DCT unit 1202, the DQT unit 1203, the quantizer 1204, and the zigzag scanning serializer 1205 sequentially perform DCT transformation, DQT transformation, quantization processing, and zigzag scanning of coefficients according to the block size calculated by the block size assignment unit 1201. The variable length coder 1206 outputs image data generated by variable length encoding the encoded block information and block size information to the variable length decoder 1208 via the transmission channel 1207. The variable length decoder 1208 receives image data via the transmission channel 1207, decodes the image data, and decodes block information and block size information. The inverse zigzag scanning serializer 1209, the inverse quantizer 1210, the IDQT unit 1211, and the IDCT unit 1212 sequentially perform inverse zigzag scanning and inverse quantization processing on the encoded data according to the block size information decoded by the variable length decoder. Then, inverse DQT processing and inverse DCT processing are performed to generate a decoded image and output it to the image output unit 1213.

Non-Patent Document 1 discloses an image compression technique that omits encoding of block size information by performing DCT conversion with the same block size as that used for motion prediction encoding.
Special table 2003-523652 publication gazette “Variable Block-Size Transforms For H.264 / AVC”, Mathias Wien, IEEE Transactions on circuits and systems for video technology VOL. 13, No.7 July 2003

  However, the method of calculating the DCT block size using the input image as in the image encoding device of Patent Document 1 described above has the following problems. That is, since it is impossible to use an input image when decoding encoded data, it is necessary to transmit block size information of DCT processing to a decoding side apparatus. Accordingly, the block size information that is the processing unit of the encoding is encoded together with the encoded data of the image, but the code amount of the encoded data of this block size information increases as the resolution of the image increases. There is a problem that encoding efficiency is reduced due to overhead of size information.

  Further, as in the technique of Non-Patent Document 1 described above, the method of setting the DCT block size using the size of the motion prediction block cannot be applied to other than the motion prediction block. It cannot be used for an intra-picture encoding method with high image quality. Furthermore, since the block size used in the motion prediction block does not reflect the frequency characteristics of the image, there is a problem that it is not possible to improve the encoding by changing the block size and performing DCT conversion.

  The present invention has been made in view of the above points, and an object of the present invention is to improve the coding efficiency by changing the block according to the image characteristics without the need for coding the block information.

  An image encoding apparatus according to the present invention includes an original image input unit that inputs an original image, a basic image acquisition unit that acquires a basic image serving as a basis of the original image, and a difference image between the original image and the basic image. Difference image calculation means to be obtained; block information acquisition means for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image; and obtaining block information indicating a division mode; And a differential image encoding means for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, block information indicating a block that is a processing unit for encoding a difference image can be acquired from a basic image that has a strong correlation with the difference image, so that block information need not be encoded. In addition, encoding efficiency can be improved by making the processing unit for encoding a differential image in which a flat portion and an edge portion are mixed variable.

  An image encoding apparatus according to another aspect of the present invention includes an original image input unit that inputs an original image, a basic image acquisition unit that acquires an image obtained by down-sampling the original image as a basic image, and the original image Difference image calculation means for obtaining a difference image from the basic image, and dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image, and obtaining block information indicating a division mode Block information acquisition means, differential image encoding means for encoding a difference image using a plurality of blocks divided in the mode shown in the block information as processing units, and basic image encoding means for encoding the basic image; It has the composition provided with.

  With this configuration, in hierarchical encoding that creates encoded data that can be adapted to the environment of a decoding-side apparatus, using a basic image (basic layer) obtained by down-sampling an original image and a differential image (enhancement layer), Since block information indicating a block which is a processing unit of differential image encoding can be acquired from the basic image, it is not necessary to encode the block information. In addition, encoding efficiency can be improved by making the processing unit for encoding a differential image in which a flat portion and an edge portion are mixed variable.

  An image encoding device according to another aspect of the present invention includes an image input unit that inputs an image composed of a plurality of continuous frames, and a basic image acquisition unit that acquires a previous frame of a frame of an original image to be encoded as a basic image. A difference image calculation means for obtaining a difference image between the original image and the basic image, and dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image, Block information acquisition means for acquiring block information indicating the difference information, and difference image encoding means for encoding a difference image using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, in an image composed of a plurality of frames, block information indicating a block that is a processing unit for encoding a difference image can be acquired from a previous frame having a strong correlation with the difference image, so that the block information is encoded. There is no need. In addition, encoding efficiency can be improved by making the processing unit for encoding a differential image in which a flat portion and an edge portion are mixed variable.

  In the image encoding device, the block information acquisition unit may acquire the block information by dividing a decoded image obtained by decoding encoded data obtained by encoding the basic image into a plurality of blocks.

  By obtaining block information from the basic image decoded in this way, the same block as at the time of decoding is taken into consideration when changing the basic image due to loss of image information accompanying encoding and decoding. Information can be acquired.

  The image coding apparatus may further include coding noise removing means for removing coding noise generated in the decoded image by coding and decoding the basic image.

  With this configuration, generation of pseudo edge noise can be suppressed and an accurate edge image can be generated. As the coding noise removing means, for example, a deblocking filter or a low-pass filter can be used.

  In the image encoding device, the block information acquisition unit may divide an edge extracted image obtained by extracting an edge of the basic image so that an edge intensity in each block is smaller than a predetermined threshold.

  With this configuration, a block including a region having a large edge strength is divided into small blocks to reduce the occurrence of pseudo contour noise near the edge, and a block not including a region having a large edge strength is divided into large blocks, The efficiency of block coding can be improved.

  In the image encoding device, a plurality of threshold values used by the block information acquisition unit may be set according to the block size.

  By setting a plurality of threshold values in this way, it is possible to set the mode of block division without being restricted by the relationship between the block size and the average edge strength.

  The image encoding apparatus includes a threshold setting unit configured to set the threshold so that the basic image is divided in the same manner as when the difference image is divided based on edge information included in the difference image. Also good.

  In this way, the difference image to be encoded is divided into processing unit blocks, and the threshold value is set so that the basic image is divided in the same manner as the division mode. The block information of the processing unit with high conversion efficiency can be acquired from the basic image.

  The image encoding device may further include threshold encoding means for encoding the threshold used by the block information acquisition means.

  With this configuration, it is possible to change the threshold according to the image characteristics, and it is possible to further improve the coding efficiency by flexibly controlling the block as a processing unit.

  The image encoding apparatus may further include an edge extraction filter encoding unit that encodes an edge extraction filter used for edge extraction by the block information acquisition unit.

  With this configuration, it is possible to select an edge extraction method in accordance with image characteristics and accurately generate an edge image that is a source of block determination for each processing unit.

  In the image encoding device, the block information acquisition unit obtains a color distribution from color difference components of the basic image, divides the basic image into a plurality of blocks based on the color distribution, and acquires the block information. Also good.

  With this configuration, a processing block with high coding efficiency can be determined based on color difference information even for an image in which an input image includes many edges and it is difficult to divide a block based on the edges.

  In the image coding apparatus, the block information acquisition unit may divide the color distribution dispersion value in each block so as to be smaller than a predetermined threshold value.

  With this configuration, even for an image that includes many edges of an input image and it is difficult to divide a block based on the edges, a region having the same color and high cross-correlation can be encoded into one block. Encoding efficiency can be improved.

  In the image encoding device, the difference image encoding means may perform orthogonal transform using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, it is possible to perform orthogonal transformation with a block suitable for image characteristics using the block information acquired by the block information acquisition means.

  In the image encoding device, the difference image encoding means may perform a discrete cosine transform as the orthogonal transform.

  According to this configuration, a large block including a large number of pixel values is converted into a DCT coefficient including a small number of non-zero effective coefficients that are concentrated on low frequency coefficients, and the small number of effective coefficients are reduced with a small amount of code. Since encoding can be performed, encoding efficiency can be improved.

  In the image encoding device, the difference image encoding means may perform discrete wavelet transform as the orthogonal transform.

  According to this configuration, a large block including a large number of pixel values is converted to a DWT coefficient including a small number of non-zero effective coefficients that are concentrated in low subband coefficients, and the small number of effective coefficients is reduced in code amount. Therefore, encoding efficiency can be improved.

  In the image encoding device, the difference image encoding means may perform entropy encoding using a plurality of blocks divided in the mode shown in the block information as processing units.

  According to this configuration, a large block including a large number of pixel values can be encoded with a small amount of code using a non-zero bias of the effective coefficient, so that the encoding efficiency can be improved.

  The image encoding device includes an output unit that outputs encoded data encoded by the difference image encoding unit, and the output unit performs the encoding for each block size of an encoding processing unit. Data may be output.

  With this configuration, since the block sizes of processing units of continuous encoded data are uniform, it is possible to efficiently transmit encoded data or decode encoded data.

  In the image encoding apparatus, the output unit may output the encoded data in order from encoded data having a larger block size in a processing unit.

  With this configuration, a block having a large size including a flat area having a large influence on the subjective image quality can be transmitted first, and the image quality can be preferentially improved.

  An image decoding apparatus according to the present invention includes encoded data input means for inputting encoded data obtained by encoding an image, and a basic unit that obtains a basic image that is a basis of an original image by decoding part of the encoded data. Block information indicating a division mode by dividing the basic image into a plurality of blocks having different sizes based on image decoding means and edge information included in the basic image decoded by the basic image decoding means And a difference image decoding unit that obtains a difference image by decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as a processing unit. .

  With this configuration, block information indicating a block that is a processing unit for decoding a difference image can be acquired from a basic image having a strong correlation with the difference image, so that encoding is efficiently performed without including block information. Data can be decoded using a block of processing units.

  An image decoding apparatus according to another aspect of the present invention includes encoded data input means for inputting encoded data obtained by encoding an image, and down-sampling processing of an original image by decoding a part of the encoded data Dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image decoded by the basic image decoding unit; Block information acquisition means for acquiring block information indicating a division mode, and differential image decoding means for decoding the encoded data using a plurality of blocks divided in the mode indicated by the block information as processing units to obtain a differential image It has the composition provided with.

  With this configuration, in hierarchical encoding that creates encoded data that can be adapted to the environment of a decoding-side apparatus, using a basic image (basic layer) obtained by down-sampling an original image and a differential image (enhancement layer), Since block information indicating a block that is a processing unit of differential image decoding can be obtained from a basic image, encoded data that is efficiently encoded without including block information can be obtained using a block of processing unit. Decryption can be performed.

  An image decoding apparatus according to another aspect of the present invention includes encoded data input means for inputting encoded data obtained by encoding an image composed of a plurality of continuous frames, and decoding to be included in the encoded data A block information acquisition means for setting a previous frame of the frame as a basic image, dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image, and acquiring block information indicating a division mode; Difference image decoding means for obtaining a difference image by decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as processing units, and an original of the frame based on the basic image and the difference image And an original image synthesis means for synthesizing images.

  With this configuration, in an image composed of a plurality of frames, block information indicating a block that is a processing unit for decoding a difference image can be acquired from a previous frame having a strong correlation with the difference image. The encoded data encoded efficiently can be decoded using a block of processing units.

  In the image decoding apparatus, the block information acquisition unit may divide an edge extracted image obtained by extracting an edge of the basic image so that an edge strength in each block is smaller than a predetermined threshold.

  With this configuration, it is possible to decode encoded data that has been efficiently encoded using blocks divided based on edge strength as processing units.

  In the image decoding apparatus, a plurality of threshold values used by the block information acquisition unit may be set according to the block size.

  With this configuration, it is possible to decode encoded data that has been efficiently encoded using blocks divided by using a plurality of thresholds as processing units.

  In the image decoding device, the threshold used by the block information acquisition unit may be input together with the encoded data.

  With this configuration, it is possible to decode image data without requiring block information even for encoded data that is efficiently encoded by changing the threshold according to the image characteristics.

  The image decoding apparatus includes an edge extraction filter decoding unit that decodes an edge extraction filter included in the encoded data, and the block information acquisition unit uses the edge extraction filter to detect an edge of the basic image. It may be extracted.

  With this configuration, it is possible to decode image data without requiring block information even for encoded data that is efficiently encoded by changing the edge extraction method according to the image characteristics.

  In the image decoding apparatus, the block information obtaining unit may obtain a color distribution from a color difference component of the basic image, and obtain the block information by dividing into a plurality of blocks based on the color distribution.

  With this configuration, it is possible to decode the encoded data that is efficiently encoded using the blocks divided based on the color distribution as processing units.

  In the image decoding apparatus, the block information obtaining unit may divide the color distribution dispersion value in each block so as to be smaller than a predetermined threshold value.

  With this configuration, it is possible to decode code data that is efficiently encoded by combining regions having the same color and high cross-correlation into one block.

  In the image decoding apparatus, the differential image decoding means may perform orthogonal transform using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, it is possible to perform a decoding process on encoded data encoded by orthogonal transform in which a processing unit block is changed.

  In the image decoding apparatus, the difference image decoding means may perform discrete cosine transform as the orthogonal transform.

  With this configuration, it is possible to perform decoding processing even on encoded data that has been efficiently encoded by DCT conversion in which processing unit blocks are changed.

  In the image decoding apparatus, the difference image decoding means may perform discrete wavelet transform as the orthogonal transform.

  With this configuration, it is possible to perform decoding processing on encoded data that has been efficiently encoded by DWT conversion in which processing unit blocks are changed.

  In the image decoding device, the difference image decoding means may perform entropy decoding using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, it is possible to perform a decoding process even on encoded data that is efficiently encoded by entropy encoding in which a block of a processing unit is changed.

  An image encoding method of the present invention includes an original image input step for inputting an original image, a basic image acquisition step for acquiring a basic image serving as a basis of the original image, and a difference image between the original image and the basic image. A difference image calculation step to be obtained; a block information acquisition step for dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image; and obtaining block information indicating a division mode; A differential image encoding step for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units.

  With this configuration, similarly to the image encoding device of the present invention, the block of the processing unit for encoding the difference image can be made variable without encoding the block information, and the encoding efficiency can be improved. Various configurations of the image coding apparatus of the present invention can also be applied to the image coding method of the present invention.

  The image decoding method of the present invention includes an encoded data input step for inputting encoded data obtained by encoding an image, and a basic method for obtaining a basic image serving as a basis of an original image by decoding a part of the encoded data. An image decoding step; dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image decoded in the basic image decoding step; and block information indicating a division mode A block information acquisition step to be acquired; and a differential image decoding step of decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as a processing unit to obtain a differential image.

  With this configuration, similarly to the image decoding method of the present invention, it is possible to decode encoded data that is efficiently encoded without including block information, using a variable processing unit block. Various configurations of the image decoding apparatus of the present invention can also be applied to the image decoding method of the present invention.

  According to the present invention, the block information indicating the block which is the processing unit of the differential image encoding is obtained from the basic image having a strong correlation with the differential image, and the block of the differential image encoding processing unit is variable, In addition, since it is not necessary to encode the block information for determining the processing unit, it has an excellent effect of improving the encoding efficiency.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the first embodiment, an example in which a hierarchical encoding method is used as an image encoding method will be described. The hierarchical coding method is a method of performing coding by dividing an input image into a base layer and an enhancement layer. The base layer can decode a single image, and the enhancement layer is obtained by encoding the interpolation data of the base layer. By having the enhancement layer, it is possible to improve the quality of the image decoded from the base layer. The enhancement layer in the present embodiment is assumed to be a resolution enhancement layer that improves resolution.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration of an image encoding device according to the first embodiment, and FIG. 2 is a diagram illustrating a configuration of an image decoding device according to the first embodiment. Prior to the description of the image encoding device using FIG. 1 and the description of the image decoding device using FIG. 2, the configuration of a video surveillance system in which the image encoding device and the image decoding device are used will be described.

  FIG. 3 is a block diagram showing a configuration of a video surveillance system using the image encoding device 100 and the image decoding device 200 according to the first embodiment of the present invention. As shown in FIG. 3, the image encoding device 100 and the image decoding device 200 are connected via a communication network 301. In FIG. 3, the image encoding device 100 encodes an image captured by the camera 302 and outputs encoded data to the communication network 301. The communication network 301 transfers the encoded data encoded by the image encoding device 100 to the image decoding device 200. The image decoding apparatus 200 decodes the encoded data received via the communication network 301 and displays an image on the display 303.

  Next, the image encoding device 100 will be described. An image encoding apparatus 100 shown in FIG. 1 includes an image input unit 101, a difference processing unit 102, an orthogonal transform unit 103, an entropy encoding unit 104, a downsampling unit 105, a base layer encoding / decoding unit 106, and an upsampling unit. 107, a block information acquisition unit 108, and an encoded data output unit 109.

  The image input unit 101 is connected to the external camera 302 and outputs image data captured by the camera 302 to the difference processing unit 102 and the downsampling unit 105 for each frame.

  The downsampling unit 105 performs a downsampling process that reduces the image data input from the image input unit 101 to a predetermined spatial resolution, and outputs the downsampled image to the base layer encoding / decoding unit 106. Here, the downsampled image is a base layer image in the hierarchical coding scheme.

  The base layer encoding / decoding unit 106 encodes the downsampled image input from the downsampling unit 105 using an encoding method such as MPEG-4, and outputs the base layer encoded data to the image data output unit 109. Also, the base layer encoding / decoding unit 106 decodes the encoded data, and outputs the decoded base layer image (base layer decoded image) to the upsampling unit 107.

  The up-sampling unit 107 performs up-sampling processing in which the base layer decoded image input from the base layer encoding / decoding unit 106 is set to a predetermined spatial resolution, and the base layer decoded image after up-sampling is subjected to a difference processing unit. 102 and the block information acquisition unit 108.

  The block information acquisition unit 108 divides the base layer decoded image after upsampling processing input from the upsampling unit 107 into a plurality of blocks based on edge information. Specifically, a region with high edge strength is divided into small blocks, and a region with low edge strength is divided into large blocks. The divided block is a processing unit for encoding and decoding. The block information acquisition unit 108 acquires block information indicating the mode of the divided blocks, and outputs the acquired block information to the orthogonal transform unit 103 and the entropy encoding unit 104.

  Here, when the base layer decoded image input from the upsampling unit 107 has a lower image quality than the input image from the image input unit 101, the block information acquisition unit 108 performs the edge extraction. A process for removing coding noise may be performed. Whether or not the image quality is low can be determined by whether or not the S / N value calculated by comparing the input image and the base layer decoded image exceeds a predetermined threshold value. By performing this process, it is possible to generate an accurate edge image while suppressing the generation of pseudo edge noise.

  The difference processing unit 102 performs difference processing in units of pixels using the image data input from the image input unit 101 and the base layer decoded image after up-sampling processing input from the up-sampling unit 107, and performs difference processing on a pixel basis. Is generated. The difference processing unit 102 outputs the generated difference image to the orthogonal transform unit 103.

  The orthogonal transform unit 103 performs a DCT (DISCLETE COSINE TRANSFORM) transform process on the difference image input from the difference processing unit 102. A block that is a processing unit of the DCT conversion process is determined based on the block information input from the block information acquisition unit 108. The orthogonal transform unit 103 outputs the DCT transform coefficient obtained by the DCT transform process to the entropy coding unit 104.

  The entropy encoding unit 104 performs entropy encoding on the DCT transform coefficient input from the orthogonal transform unit 103. A block serving as a processing unit of the entropy encoding process is determined based on the block information input from the block information acquisition unit 108. The entropy encoding unit 104 outputs the entropy-encoded encoded data to the encoded data output unit 109.

  The encoded data output unit 109 is connected to the communication network 301, and the base layer encoded data input from the base layer encoding / decoding unit 106 and the enhancement layer encoding input from the entropy encoding unit 104. The data is multiplexed and output to the communication network 301.

  Next, the image decoding apparatus 200 will be described. As shown in FIG. 2, the image decoding apparatus 200 includes an encoded data input unit 201, an entropy decoding unit 202, an orthogonal transformation unit 203, an image addition unit 204, a base layer decoding unit 205, a block An information acquisition unit 206, an upsampling unit 207, and an image output unit 208 are included.

  The encoded data input unit 201 is connected to the communication network 301 and receives encoded data transmitted from the image encoding device 100 via the communication network 301. The encoded data input unit 201 detects the start code of each base / enhancement layer from the encoded data, outputs the base layer encoded data to the base layer decoding unit 205, and entropy the enhancement layer encoded data. The data is output to the decoding unit 202.

  The base layer decoding unit 205 decodes the encoded data input from the encoded data input unit 201 by a decoding method such as MPEG-4 to generate a base layer decoded image, and the upsample unit 207 Output to.

  The upsampling unit 207 performs upsampling processing for converting the base layer decoded image input from the base layer decoding unit 205 into a preset spatial resolution, and the base layer decoded image after the upsampling processing is converted into block information. The data is output to the acquisition unit 206 and the image addition unit 204.

  The block information acquisition unit 206 divides the upsampled base layer decoded image input from the upsampling unit 207 into a plurality of blocks based on the edge information, and acquires block information indicating a division mode. The block divided here becomes a decoding processing unit. The block information acquisition unit 20 outputs the acquired block information to the orthogonal transform unit 203 and the entropy decoding unit 202.

  The entropy decoding unit 202 performs entropy decoding on the enhancement layer encoded data input from the encoded data input unit 201 to generate a DCT transform coefficient. The block that is the processing unit of the entropy decoding process is determined based on the block information input from the block information acquisition unit 206. The entropy decoding unit 202 outputs the decoded DCT transform coefficient to the orthogonal transform unit 203.

  The orthogonal transform unit 203 performs an inverse DCT transform process on the DCT transform coefficient input from the entropy decoding unit 202 to generate a differential decoded image. The block that is the processing unit of the inverse DCT conversion process is determined based on the block information input from the block information acquisition unit 206. The orthogonal transform unit 203 outputs the generated differentially decoded image to the image addition unit 204.

  The image addition unit 204 adds the base layer decoded image after up-sampling processing input from the up-sampling unit 207 and the differential decoded image input from the orthogonal transformation unit 203 in a pixel unit, thereby decoding the decoded image. Is output to the image output unit 208.

  The image output unit 208 is connected to the display 303 and outputs the decoded image input from the image addition unit 204 to the display.

  Next, the operation of the image coding apparatus 100 having the above configuration will be described using the flowchart shown in FIG. The operation of the flowchart shown in FIG. 4 is stored as a control program in a storage device (not shown) of the image encoding device 100 (for example, a ROM or a flash memory) and is controlled by a CPU (not shown).

  First, the image input unit 101 of the image encoding device 100 performs input processing of a captured image from the connected camera 302 (S10). Specifically, the image input unit 101 outputs the input image data to the difference processing unit 102 and the downsampling unit 105 for each frame.

  Next, the down-sampling unit 105 performs a down-sampling process that reduces the image data input from the image input unit 101 to a predetermined spatial resolution (S12), and performs base layer encoding / decoding of the image after the down-sampling process. To the conversion unit 106. In this embodiment, as an example of downsampling, the spatial resolution of the input image is halved both horizontally and vertically. That is, if the spatial resolution of the input image is (X pixels × Y pixels), the spatial resolution after the down-sampling process is (X / 2 pixels × Y / 2 pixels). As for the down-sampling method, any method can be applied as long as the spatial resolution is changed. For example, it is possible to use subband division using the WAVELET transform.

  Next, the base layer encoding / decoding unit 106 encodes the down-sampled image input from the down-sampling unit 105 using an encoding method such as MPEG-4, and the base layer encoded data is sent to the image data output unit 109. Output (S14). In addition, base layer encoding / decoding section 106 decodes the encoded data and outputs the base layer decoded image to upsampling section 107. Here, as the base layer encoding method, any encoding method other than MPEG-4 can be used.

  Next, the block information acquisition unit 108 uses the base layer decoded image after the upsampling process input from the upsampling unit 107 to acquire block information indicating a block division mode (S16). Here, the block information acquisition process will be described in detail.

  FIG. 5 is a diagram showing a flow of block information acquisition processing. First, the block information acquisition unit 108 performs edge extraction processing on the base layer decoded image to generate an edge image (S30). In the present embodiment, the edge extraction processing is realized by performing edge detection filter processing using, for example, a SOBEL operator.

  Next, the block information acquisition unit 108 calculates the edge strength with respect to the edge image of the base layer in units of a large block composed of a certain number of pixels (S32). Here, the large block is a lattice block composed of 32 × 32 pixels, the middle block is a lattice block composed of 16 × 16 pixels, and the small block is a lattice block composed of 8 × 8 pixels. Suppose there is.

図６（ａ）〜図６（ｃ）は、フレーム内におけるブロックの処理順序を説明するための図である。図６（ａ）において、フレーム６０１は画像の１フレームを示し、ブロック６０２は３２×３２画素の大ブロックを示す。図６（ａ）に示すように左上のブロック６０２ａから閾値処理を開始し、順次右側のブロックへと処理を行っていく。一番右側のブロックの処理が終了すると、一段下の行の左側のブロックから右側に処理を行っていく。そして、右下のブロック６０２ｂの閾値処理が終了すると、１フレームの画像のブロック情報が取得される。 The edge intensity AE (i) of the large block i is expressed as follows: (x, y) ∈ AREA (i) is a set of positions of pixels included in the large block i, N is the number of pixels in the large block, Edge (x, y ) As a pixel value of the edge image at the pixel position (x, y), is calculated by the following equation (1).

FIG. 6A to FIG. 6C are diagrams for explaining the processing order of blocks in a frame. In FIG. 6A, a frame 601 indicates one frame of an image, and a block 602 indicates a large block of 32 × 32 pixels. As shown in FIG. 6A, threshold processing is started from the upper left block 602a, and processing is sequentially performed to the right block. When the processing of the rightmost block is completed, the processing is performed from the left block in the lower row to the right. Then, when the threshold processing of the lower right block 602b ends, block information of an image of one frame is acquired.

  Next, the edge strength is compared with the large block threshold value (S34). Specifically, the edge strength of the large block and the large block threshold (TH_1) are compared. If true, the block size of the large block i is determined to be the large block (S36), and the process proceeds to the end determination process (S48). , AE (i) <TH_1 is false, the process of step S38 is performed.

  Next, when the determination of the large block threshold is false, the block information acquisition unit 108 calculates the edge strength of the medium block (S38). Specifically, as shown in FIG. 6B, the large block 602 includes four medium blocks 603a to 603d. Edge strength is calculated for each of the middle blocks 603a to 603d.

本処理ステップにおいては、大ブロック６０２を分割して構成される４つの中ブロック６０３ａ〜６０３ｄに対して、左上の中ブロック６０３ａからエッジ強度算出を開始し、右上の中ブロック６０３ｂ、左下の中ブロック６０３ｃ、右下の中ブロック６０３ｄの順で処理を行っていく。 The edge strength AE (i) of the j-th middle block of the large block i is (x, y) ∈AREA (i) as a set of positions of pixels included in the middle block j of the large block i, and N is the middle block The number of pixels, Edge (x, y), is calculated as the pixel value of the edge image at the pixel position (x, y) by the following equation (2).

In this processing step, edge strength calculation is started from the upper left middle block 603a for the four middle blocks 603a to 603d configured by dividing the large block 602, and the upper right middle block 603b and the lower left middle block are started. Processing is performed in the order of 603c and lower right middle block 603d.

  Subsequently, the block information acquisition unit 108 compares the edge strength with the middle block threshold (S40). Specifically, the edge strength of the middle block is compared with the middle block threshold value (TH_2), and if the expression AE (i, j) <TH_2 is true, the block size of the middle block j is determined as the middle block (S42). ), And moves to the process of step S46. Conversely, if the expression AE (i, j) <TH_2 is false, the block size of the medium block j is determined to be a small block (S44). That is, as shown in FIG. 6C, after the middle block 603 is divided into four small blocks of 8 × 8 pixels, the process proceeds to step S46.

  In step S46, the block information acquisition unit 108 performs the middle block end determination. Specifically, it is determined whether edge strength calculation processing and threshold determination processing have been performed on the four middle blocks 603a to 603d in the large block 602. If all the middle blocks have been processed, the process moves to step S48. If not, the next middle block is processed, and the process in step S38 is performed.

  Finally, the block information acquisition unit 108 makes an end determination (S48). Specifically, it is determined whether edge strength calculation and threshold determination processing have been performed on all large blocks in one frame. If all large block processes have been completed, the process ends. If not, the process moves to step S32 with the next large block as the processing target. After all large blocks have been processed, block information calculated for all large blocks in one frame is output to the orthogonal transform unit 103.

  Here, it is assumed that the two threshold values used for block determination are TH_1 <TH_2, and that the larger block is selected as the edge strength is smaller. By setting the threshold value in this way, the processing unit can be set to a larger block size in a region where the edge strength is small and the low frequency coefficient is expected to be relatively large. The number of threshold processes for determining a block is not limited to two, and the block size is not limited to three, large, medium, and small. Moreover, it is not limited to square blocks having the same vertical and horizontal sizes.

  When the acquisition of the block information is completed, the block information acquisition processing unit 108 outputs the block information to the orthogonal transform unit 103 and the entropy encoding unit 104.

  Next, returning to FIG. 4, the image encoding device 100 performs enhancement layer encoding processing (S18). Specifically, the difference processing unit 102 performs difference processing on a pixel basis on the image data input from the image input unit 101 and the base layer decoded image after up-sampling processing input from the up-sampling unit 107, The difference image is output to the orthogonal transform unit 103. Note that the difference image calculated here originally exists in the input image data and includes many image components reduced in the base layer encoding, that is, many edge components that are easily lost in the encoding process. It is an image with high correlation. Accordingly, the block information calculated using the edge image of the base layer is similar to the block information calculated using the difference image.

  The orthogonal transform unit 103 performs DCT (DISCLETE COSINE TRANSFORM) transform processing on the difference image input from the difference processing unit 102. Here, the DCT conversion process is performed using the block indicated by the block information input from the block information acquisition unit 108 as a processing unit. Specifically, the DCT conversion process is sequentially performed on all large blocks in the difference image with the block indicated by the block information calculated by the block information acquisition unit 108 as a processing unit. When the large block is divided into a plurality of medium blocks, and when the large block is further divided into small blocks, the DCT conversion process is performed using these medium blocks or small blocks as processing units. The orthogonal transform unit 103 outputs the DCT transform coefficients for one frame to the entropy coding unit 104. Here, the orthogonal transform is not limited to DCT, and DWT (DISCLETE WAVELET TRANSFORM) can also be used.

  The entropy encoding unit 104 performs entropy encoding on the DCT transform coefficient input from the orthogonal transform unit 103 using the block indicated by the block information input from the block information acquisition unit 108, and converts the enhancement layer encoded data into The image is output to the image output unit 109.

  Next, the encoded data output unit 109 of the image encoding device 100 outputs encoded data (S20). Specifically, the image output unit 109 multiplexes the base layer encoded data input from the base layer encoding / decoding unit 106 and the enhancement layer encoded data input from the entropy encoding unit 104, and connects Output to the communication network 301.

  Next, the image coding apparatus 100 performs end determination (S22). Specifically, when the image input from the camera 302 is completed, or when a process of a predetermined number of frames or more is performed, the process is terminated. Otherwise, the process moves to step S10.

  Next, the operation of the image decoding apparatus 200 will be described using the flowchart shown in FIG. The operation of the flowchart shown in FIG. 7 is stored as a control program in a storage device (not shown) such as a ROM or a flash memory of the image decoding device 200, and is controlled by a CPU (not shown).

  The image decoding apparatus 200 first inputs encoded data (S50). Specifically, the encoded data input unit 201 receives encoded data for one frame transmitted from the image encoding device 100 via the communication network 301. Then, the image decoding apparatus 200 detects the base / enhancement layer start codes from the received encoded data, outputs the base layer encoded data to the base layer decoding unit 205, and performs enhancement layer encoding. The data is output to the entropy decoding unit 202.

  Next, the image decoding apparatus 200 performs decoding of the base layer (S52). Specifically, the base layer decoding unit 205 decodes the base layer encoded data input from the encoded data input unit 201 by a decoding method such as MPEG-4, and generates a base layer decoded image. Is generated and output to the up-sampling unit 207.

  Subsequently, the upsampling unit 207 upsamples the base layer decoded image input from the base layer decoding unit 205 to a preset spatial resolution, and converts the base layer decoded image after the upsampling into block information. The data is output to the acquisition unit 206 and the image addition unit 204.

  Next, the block information acquisition unit 206 performs block information acquisition processing (S54). Specifically, the block information acquisition unit 206 divides the base layer decoded image after up-sampling processing input from the up-sampling unit 207 based on edge information, and acquires block information indicating a division mode. . Here, the block information acquisition unit 206 performs the same processing as the block information acquisition unit 108 of the image encoding device 100 (see FIG. 5). The block information acquisition unit 206 outputs the acquired block information to the entropy decoding unit 202 and the orthogonal transform unit 203.

  After acquiring the block information, the image decoding apparatus 200 performs enhancement layer decoding (S56). Specifically, the entropy decoding unit 202 entropy-decodes the enhancement layer encoded data input from the encoded data input unit 201 to generate a DCT transform coefficient. Here, the processing unit of entropy decoding is a block indicated by the block information input from the block information acquisition unit 206. The entropy decoding unit 202 outputs the decoded DCT transform coefficient to the orthogonal transform unit 203.

  Subsequently, the orthogonal transform unit 203 performs an inverse DCT transform process on the DCT transform coefficient input from the entropy decoding unit 202 to generate a differential decoded image. Here, the processing unit of the inverse DCT transform process is a block indicated by the block information input from the block information acquisition unit 206. The orthogonal transform unit 203 outputs the generated differentially decoded image to the image addition unit 204.

  Subsequently, the image addition unit 204 adds the base layer decoded image after up-sampling processing input from the up-sampling unit 207 and the differential decoded image input from the orthogonal transformation unit 203 in units of pixels to perform image synthesis. Processing is performed to generate a decoded image (S58). The image addition unit 204 outputs the generated decoded image to the image output unit 208.

  The image output unit 208 outputs the decoded image input from the image addition unit 204 (S60). Specifically, the image output unit 208 outputs the decoded image input from the image addition unit 204 to the display.

  Next, the image decoding apparatus 200 performs an end determination (S62). Specifically, if reception of encoded data from the communication network 301 is stopped, or if decoding processing of a predetermined number of frames or more has been performed, the processing ends, and if not, the processing proceeds to step S50. Move to.

  Heretofore, the image encoding device 100 and the image decoding device 200 according to the first embodiment have been described.

  In image coding apparatus 100 according to the present embodiment, block information acquisition section 103 of image coding apparatus 100 uses a decoded image obtained by decoding encoded base layer encoded data, and is a block that is a processing unit. By acquiring the block information indicating the block information, it is possible to change the block of the processing unit of the enhancement layer encoding in the hierarchical encoding without requiring the encoding of the block information. Thereby, the encoding efficiency of an image can be improved.

  Also, in the image decoding apparatus 200 according to the present embodiment, the block information acquisition unit 206 performs encoding by acquiring block information using a base layer decoded image obtained by decoding base layer encoded data. Without including block information in the data, it is possible to decode the enhancement layer image data encoded efficiently by changing the block.

(Second Embodiment)
Next, an image encoding device and an image decoding device according to the second embodiment of the present invention will be described. In the second embodiment, an example is shown in which a normal encoding method is used as an image encoding method instead of a hierarchical encoding method. Also in the second embodiment, as in the first embodiment, the image encoding device and the image decoding device are applied to a video surveillance system (see FIG. 3).

  FIG. 8 is a block diagram showing a configuration of an image coding apparatus 800 according to the second embodiment of the present invention. As illustrated in FIG. 8, the image encoding device 800 includes an image input unit 801, a motion prediction unit 802, an orthogonal transform unit 803, an entropy encoding unit 804, an encoded data output unit 805, a motion compensation unit 806, and block information acquisition. Part 807.

  The image input unit 801 is connected to the external camera 302, and outputs image data captured by the camera 302 to the motion prediction unit 802 for each frame.

  The motion prediction unit 802 compares the image data input from the image input unit 801 with the decoded image of the previous frame input from the motion compensation unit 806 (defined as a reference image) for each block of 16 × 16 pixels. A motion vector to a position where the correlation between the input image data and the reference image is high is calculated. Subsequently, after the input image data is moved in the direction of the motion vector, a difference process with the reference image is performed to generate a difference image and output to the orthogonal transform unit 803. Furthermore, the motion prediction unit 802 outputs the motion vector to the motion compensation unit 806 and the entropy encoding unit 804.

  The orthogonal transform unit 803 performs DCT (DISCLETE COSINE TRANSFORM) transform on the difference image input from the motion prediction unit 802 to generate a DCT coefficient. Here, the DCT conversion process is performed using the block input from the block information acquisition unit 807 and indicated in the block information as a processing unit. The orthogonal transform unit 803 outputs the generated DCT coefficient to the entropy coding unit 804, performs inverse DCT transform on the DCT coefficient, decodes the difference image, and outputs the decoded difference image to the motion compensation unit 806.

  The entropy encoding unit 804 performs entropy encoding processing on the DCT coefficient input from the orthogonal transform unit 803. Here, entropy encoding is performed using the block indicated by the block information input from the block information acquisition unit 807 as a processing unit. The entropy encoding unit 804 generates encoded data by encoding the motion vector input from the motion prediction unit 802. The generated image and motion vector encoded data is output to the encoded data output unit 805.

  The motion compensation unit 806 has a frame buffer therein and stores a decoded image one frame before. Then, the motion compensation unit 806 moves the decoded differential image input from the orthogonal transform unit 803 in the direction of the motion vector input from the motion prediction unit 802, and then performs addition processing with the image in the frame buffer. By doing so, a decoded image is generated. Then, the motion compensation unit 806 outputs the generated decoded image to the motion prediction unit 802 and the block information acquisition unit 807, and replaces the data in the frame buffer with a newly generated decoded image.

  The block information acquisition unit 807 acquires block information using the decoded image input by the motion compensation unit 806, and outputs the acquired block information to the orthogonal transform unit 803 and the entropy encoding unit 804.

  The encoded data output unit 805 is connected to the communication network 301 and outputs the encoded data input by the entropy encoding unit 804 to the communication network 301.

  FIG. 9 is a block diagram illustrating a configuration of an image decoding apparatus 900 according to the second embodiment. As illustrated in FIG. 9, the image decoding apparatus 900 includes an encoded data input unit 901, an entropy decoding unit 902, an orthogonal transform unit 903, a motion compensation unit 904, an image output unit 905, and a block information acquisition unit 906.

  The encoded data input unit 901 is connected to the communication network 301, receives the encoded data output from the image encoding apparatus 800 via the communication network 301, detects a start code from the encoded data, and detects one frame. Are output to the entropy decoding unit 902.

  The entropy decoding unit 902 performs entropy decoding processing on the encoded data input from the encoded data input unit 901 to generate DCT transform coefficients. Here, the entropy decoding process is performed using the block indicated by the block information input from the block information acquisition unit 906 as a processing unit. The entropy decoding unit 902 outputs the generated DCT transform coefficient to the orthogonal transform unit 903, decodes a motion vector from the encoded data, and outputs the decoded motion vector to the motion compensation unit 904.

  The orthogonal transform unit 903 performs an inverse DCT transform process on the DCT transform coefficient input from the entropy decoding unit 902 to generate a differential decoded image. Here, the inverse DCT conversion process is performed using the block indicated by the block information input from the block information acquisition unit 906 as a processing unit. The orthogonal transform unit 903 outputs the generated differentially decoded image to the motion compensation unit 904.

  The block information acquisition unit 906 divides the decoded image input from the motion compensation unit 904 based on edge information, and acquires block information indicating a division mode. The block information acquisition unit 906 outputs the acquired block information to the orthogonal transform unit 903 and the entropy decoding unit 901.

  The motion compensation unit 904 has a frame buffer therein and stores a decoded image one frame before. Then, the decoded differential image input from the orthogonal transform unit 903 is decoded by moving the image in the direction of the motion vector input from the entropy decoding unit 902 and then performing addition processing with the image in the frame buffer. Generate an image. Then, the motion compensation unit 904 outputs the generated decoded image to the image output unit 905 and the block information acquisition unit 807, and replaces the data in the frame buffer with the newly generated decoded image.

  The image output unit 905 is connected to the display 303 and outputs the decoded image input from the motion compensation unit 904 to the display 303.

  Next, the operation of the image coding apparatus 800 will be described using the flowchart shown in FIG. Note that the operation of the flowchart shown in FIG. 10 is stored as a control program in a storage device (for example, a ROM or a flash memory) (not shown) of the image encoding device 800 and is controlled by a CPU (not shown).

  First, the image input unit 801 of the image encoding device 800 inputs image data captured by the connected camera 302 for each frame (S70). The image encoding apparatus 800 determines whether or not the input frame is the first frame (S72). If the input frame is the first frame, the image encoding apparatus 800 performs the process of step S74 and is not the first frame. Performs the process of step S76.

  When the encoded frame is the first frame, there is no decoded image one frame before, so the block information acquisition unit 807 selects a fixed-size block (S74). Specifically, the block information acquisition unit 807 determines all block sizes that are encoding processing units as fixed blocks, for example, small blocks of 8 × 8 pixels, and uses the determined block size information for the orthogonal transform unit 803 and the entropy code. To the conversion unit 804.

  When the encoded frame is not the first frame, the block information acquisition unit 807 acquires block information using the decoded image of the previous frame input from the motion compensation unit 806, and converts the block information to the orthogonal transform unit 803. The data is output to the entropy encoding unit 804. The block information acquisition method according to the present embodiment is basically the same as that of the block information acquisition unit 108 according to the first embodiment, and the block information is acquired by performing threshold processing after edge extraction of the decoded image. The difference from the first embodiment is that a decoded image of the base layer is used in the first embodiment, whereas a decoded image of one frame before is used in the second embodiment. .

  Next, the image encoding device 800 performs an encoding process (S78). Specifically, the motion prediction unit 802 compares the image data input from the image input unit 801 with the decoded image of the previous frame input from the decoding unit 806 (hereinafter referred to as a reference image), and the 16 × A motion vector to a position where the correlation between the input image data and the reference image is high is calculated for each block of 16 pixels. Subsequently, after the input image data is moved in the direction of the motion vector, difference processing with the reference image is performed, a difference image is generated, and output to the orthogonal transform unit 803. Further, the motion prediction unit 802 outputs the motion vector to the motion compensation unit 806 and the entropy encoding unit 804.

  Subsequently, the orthogonal transform unit 803 performs DCT (Discrete Cosine Transform) transform on the difference image input from the motion prediction unit 802. A block that is a processing unit of the DCT conversion process is determined based on the block information input from the block information acquisition unit 108. The orthogonal transform unit 803 outputs the DCT coefficient to the entropy coding unit 804, performs inverse DCT transform on the DCT coefficient, decodes the difference image, and outputs the decoded difference image to the motion compensation unit 806. In order to reduce the DCT coefficient value, it is also possible to perform a quantization process in which the DCT coefficient is replaced by a quotient divided by the quantization parameter after the DCT conversion. When the quantization process is performed, the inverse quantization process for replacing the DCT coefficient with the multiplication result of the quantization parameter is performed before the inverse DCT conversion process.

  Subsequently, the entropy encoding unit 804 performs entropy encoding on the DCT coefficient input from the orthogonal transform unit 803. The processing unit of entropy encoding is a block indicated by the block information input from the block information acquisition unit 807. The entropy encoding unit 804 generates encoded data by encoding the motion vector input from the motion prediction unit 802 and outputs the encoded data to the encoded data output unit 805.

  The motion compensation unit 806 has a frame buffer therein and stores a decoded image one frame before. Then, the decoded differential image input from the orthogonal transform unit 803 is moved in the direction of the motion vector input from the motion prediction unit 802, and then the decoded image is added to the image in the frame buffer. Generate. Then, the generated decoded image is output to the motion prediction unit 802 and the block information acquisition unit 807, and the data in the frame buffer is replaced with a newly generated decoded image.

  The image encoding device 800 outputs encoded data (S80). Specifically, the encoded data output unit 805 outputs the encoded data input from the entropy encoding unit 804 to the connected communication network 301.

  After outputting the encoded data, the image encoding device 800 makes an end determination (S82). Specifically, when the image input from the camera 302 is completed, or when the processing of a predetermined number of frames or more has been performed, the processing is terminated. Otherwise, the processing moves to step S70.

  Next, the operation of the image decoding apparatus 900 will be described using the flowchart shown in FIG. The operation of the flowchart shown in FIG. 11 is stored as a control program in a storage device (not shown) such as a ROM or a flash memory of the image decoding apparatus 900, and is controlled by a CPU (not shown).

  The image decoding apparatus 900 first inputs encoded data (S90). Specifically, the encoded data input unit 901 receives encoded data for one frame transmitted from the image encoding apparatus 800 via the communication network 301 and outputs the encoded data to the entropy decoding unit 902.

  Next, the image decoding apparatus 900 performs the process of step S94 when the current frame to be decoded is the first frame, and performs the process of step S96 when it is not the first frame.

  When the input frame is the first frame, there is no decoded image one frame before, so the block information acquisition unit 906 selects a fixed block size. Specifically, the block information acquisition unit 906 determines all blocks that are decoding processing units as small blocks having a fixed size, for example, 8 × 8 pixels, and determines the determined block information using the orthogonal transform unit 903 and the entropy decoding unit. Output to 902.

  When the input block is not the first frame, the block information acquisition unit 906 divides the decoded image of the previous frame input from the motion compensation unit 804 based on the edge information, and shows a division mode Get information. The block information acquisition method according to the present embodiment is basically the same as that of the block information acquisition unit 206 according to the first embodiment, and after the edge of the decoded image is extracted, threshold processing is performed to determine the block information. The difference from the first embodiment is that a decoded image of the base layer is used in the first embodiment, whereas a decoded image of one frame before is used in the second embodiment. . Then, the block information acquisition unit 906 outputs the acquired block information to the orthogonal transform unit 903 and the entropy decoding unit 902.

  Next, the image decoding apparatus 900 performs a decoding process (S98). Specifically, the entropy decoding unit 902 performs entropy decoding on the encoded data input from the encoded data input unit 901 to generate a DCT transform coefficient. The processing unit for entropy decoding is a block indicated by the block information input from the block information acquisition unit 906. The entropy decoding unit 902 outputs the generated DCT transform coefficient to the orthogonal transform unit 903, decodes a motion vector from the encoded data, and outputs the decoded motion vector to the motion compensation unit 904.

  Subsequently, the orthogonal transform unit 903 performs an inverse DCT transform process on the DCT transform coefficient input from the entropy decoding unit 902 to generate a differential decoded image. The processing unit of the inverse DCT conversion process is a block indicated by the block information input from the block information acquisition unit 906. The orthogonal transform unit 903 outputs the generated differentially decoded image to the motion compensation unit 904.

  The motion compensation unit 904 moves the image in the direction of the motion vector input from the entropy decoding unit 902 from the decoded differential image input from the orthogonal transform unit 903, and then stores the previous frame stored in the frame buffer. A decoded image is generated by performing addition processing on the image (S100). Then, the motion compensation unit 904 outputs the generated decoded image to the image output unit 905 and the block information acquisition unit 807, and replaces the data in the frame buffer with the newly generated decoded image.

  The image output unit 905 outputs the decoded image input from the motion compensation unit 904 to the display (S102). Next, the image decoding apparatus 200 performs an end determination (S104). Specifically, when the reception of the encoded data from the communication network 301 is stopped, or when the decoding process of the predetermined number of frames or more is performed, the process is terminated, and when not, the process is performed in step S90. Move to. The configuration and operation of the image encoding device and the image decoding device according to the present embodiment have been described above.

  In the image encoding device 800 according to the second embodiment, the block information acquisition unit 806 acquires block information indicating a block that is a unit of encoding processing by using the decoded image of the previous frame, thereby obtaining a block. It is possible to encode a variable block as a processing unit without requiring information encoding, and to improve encoding efficiency.

  Also, in the image decoding apparatus 900 according to the present embodiment, the block information acquisition section 906 performs encoding by acquiring block information using a base layer decoded image obtained by decoding base layer encoded data. Without including block information in the data, it is possible to decode the enhancement layer image data encoded efficiently by changing the block.

  The image coding apparatus and the image decoding apparatus according to the present invention have been described in detail with reference to the embodiments. However, the present invention is not limited to the above-described embodiments.

  In the first embodiment described above, a predetermined SOBEL filter is used as the edge extraction method of the base layer decoded image by the block information acquisition unit 103 of the image encoding device 100. However, the edge extraction method is appropriately selected. It may be changed. This makes it possible to select an edge extraction method according to the image characteristics and accurately generate an edge image from which a block is determined. In this case, the entropy encoding unit 104 may generate encoded data obtained by encoding the information on the edge extraction method. Thereby, in the image decoding apparatus 200, the edge extraction of a base layer decoded image can be performed according to the edge extraction method obtained by decoding encoded data.

  In the first embodiment described above, the block information acquisition unit 103 of the image encoding device 100 performs the edge image of the base layer decoded image and acquires the block information by threshold processing. It is also possible to acquire block information using a color difference image. As a result, it is possible to acquire a processing block with high coding efficiency based on the color difference information even for an image in which the input image includes many edges and it is difficult to acquire block information from the edge image. For example, the block information acquisition unit 103 of the image encoding device 100 obtains a variance value for each block with respect to the color difference image of the base layer decoded image, and determines a larger block size for a block having a smaller variance value by threshold processing. It is also possible to do. As a result, even for images that contain many edges of the input image and it is difficult to obtain block information using the edges, the same color and high cross-correlation area can be encoded into one block. Efficiency can be improved. In this case, also in the image decoding apparatus 200, block information is acquired using a color difference image, similarly to the image encoding apparatus 100. Thereby, the same block information can be acquired by the image encoding device 100 and the image decoding device 200.

  In the first embodiment described above, the orthogonal transform unit 103 of the image encoding device 100 performs DCT transform according to the block information acquired by the block information acquisition unit 108, but it is also possible to perform DWT transform. is there. As a result, for a region with few edges, a large block including a large number of pixel values is converted into a DWT coefficient including a small number of effective coefficients concentrated on a low subband coefficient and encoded with a small code amount. Therefore, encoding efficiency can be improved.

  In the first embodiment described above, the encoded data output unit 119 of the image encoding device 100 outputs the encoded data input from the entropy encoding unit 104 to the communication network 301 in the input order. However, it is also possible to output to the communication network 301 collectively for each block size. Thereby, since the block sizes of continuous encoded data are uniform, it is possible to efficiently transmit the encoded data or decode the encoded data. For example, if the block size is output first to the communication network 301, it is possible to preferentially improve the image quality for a flat area that has a large influence on the subjective image quality.

  In the first embodiment described above, the block information acquisition unit 103 performs the threshold processing on the edge strength using the preset thresholds TH_1 and TH_2, but determines the threshold according to the image characteristics. It is good. For example, in order to determine the threshold value according to the image characteristics, the following configuration can be adopted. First, a differential image to be encoded in units of blocks is divided according to edge strength. The block information obtained by dividing the difference image is block information suitable for encoding the difference image. Next, a threshold value for dividing the base layer decoded image is set in the same manner as the blocks divided based on the difference image. As a result, since the DCT coefficient value obtained by DCT processing of the difference image becomes smaller, the encoding efficiency can be further improved. In addition, when such a structure is employ | adopted, the entropy encoding part 104 encodes a threshold value as encoding data. Thereby, the threshold set according to the image characteristic can be notified to the image decoding apparatus.

  The modification of the first embodiment described above has been described above, but the above modification can also be applied to the second embodiment.

  As described above, according to the present invention, the block information of the block, which is the processing unit of the differential image encoding, is acquired from the basic image having a strong correlation with the differential image. Since the block information is variable and it is not necessary to encode the block information that determines the processing unit, it has an excellent effect that the encoding efficiency can be improved. It can be used for applications such as conversion.

The block diagram which shows the structure of the image coding apparatus which concerns on 1st Embodiment. The block diagram which shows the structure of the image decoding apparatus which concerns on 1st Embodiment. The block diagram which shows the whole structure of the video surveillance system with which 1st Embodiment is applied. The flowchart which shows the process of the image coding apparatus which concerns on 1st Embodiment The flowchart which shows the block information acquisition process which concerns on 1st Embodiment (A)-(c) is a figure for demonstrating the concept of the block which concerns on 1st Embodiment, and the process order of a block The flowchart which shows the process of the image decoding apparatus which concerns on 1st Embodiment The block diagram which shows the structure of the image coding apparatus which concerns on 2nd Embodiment. The block diagram which shows the structure of the image decoding apparatus which concerns on 2nd Embodiment. The flowchart which shows the process of the image coding apparatus which concerns on 2nd Embodiment. The flowchart which shows the process of the image coding apparatus which concerns on 2nd Embodiment. The block diagram showing the structure of the conventional image coding apparatus and image decoding apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 101 Image input part 102 Difference processing part 103 Orthogonal transformation part 104 Entropy encoding part 105 Downsampling part 106 Base layer encoding / decoding part 107 Upsampling part 108 Block information acquisition part 109 Encoded data output part DESCRIPTION OF SYMBOLS 200 Image decoding part 201 Encoded data input part 202 Entropy decoding part 203 Orthogonal transformation part 204 Image addition part 205 Base layer decoding part 207 Upsampling part 206 Block information acquisition part 208 Image output part 301 Communication network 302 Camera 303 Display 601 Frame 602 Large block 603 Medium block 604 Small block 801 Image input unit 802 Motion prediction unit 803 Orthogonal transformation unit 804 Entropy encoding unit 805 Encoded data output unit 806 Motion compensation unit 807 Block Information acquisition unit 900 Image encoding device 901 Encoded data input unit 902 Entropy decoding unit 903 Orthogonal transformation unit 904 Motion compensation unit 905 Image output unit 906 Block information acquisition unit 1200 Image input unit 1201 Block size allocation unit 1202 DCT unit 1203 DQT Unit 1204 quantizer 1205 zigzag scanning serializer 1206 variable length coder 1207 transmission channel 1208 variable length decoder 1209 inverse zigzag scanning serializer 1210 inverse quantizer value 1211 IDQT unit 1212 IDCT unit 1213 image output unit

Claims (33)

An original image input means for inputting an original image;
Basic image acquisition means for acquiring a basic image as a basis of the original image;
Difference image calculation means for obtaining a difference image between the original image and the basic image;
Block information acquisition means for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image, and acquiring block information indicating a division mode;
Differential image encoding means for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units;
An image encoding device comprising: An original image input means for inputting an original image;
Basic image acquisition means for acquiring an image obtained by down-sampling the original image as a basic image;
Difference image calculation means for obtaining a difference image between the original image and the basic image;
Block information acquisition means for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image, and acquiring block information indicating a division mode;
Differential image encoding means for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units;
Basic image encoding means for encoding the basic image;
An image encoding device comprising: Image input means for inputting an image composed of a plurality of continuous frames;
Basic image acquisition means for acquiring the previous frame of the frame of the original image to be encoded as a basic image;
Difference image calculation means for obtaining a difference image between the original image and the basic image;
Block information acquisition means for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image, and acquiring block information indicating a division mode;
Differential image encoding means for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units;
An image encoding device comprising:   The said block information acquisition means divides | segments the decoded image obtained by decoding the encoding data which encoded the said basic image into several blocks, and acquires the said block information. Image encoding device.   The image coding apparatus according to claim 4, further comprising coding noise removing means for removing coding noise generated in the decoded image due to coding and decoding of the basic image.   The image code according to any one of claims 1 to 3, wherein the block information acquisition unit divides the edge extracted image obtained by extracting the edge of the basic image so that the edge strength in each block is smaller than a predetermined threshold. Device.   The image encoding device according to claim 6, wherein a plurality of threshold values used in the block information acquisition unit are set in accordance with a block size.   The image according to claim 6, further comprising: a threshold setting unit configured to set the threshold so that the basic image is divided in the same manner as when the difference image is divided based on edge information included in the difference image. Encoding device.   The image coding apparatus according to any one of claims 6 to 8, further comprising threshold coding means for coding the threshold used by the block information acquisition means.   The image encoding apparatus according to claim 6, further comprising an edge extraction filter encoding unit that encodes an edge extraction filter used for edge extraction by the block information acquisition unit.   The block information acquisition unit obtains a color distribution from a color difference component of the basic image, and acquires the block information by dividing the basic image into a plurality of blocks based on the color distribution. The image encoding device described in 1.   The image coding apparatus according to claim 11, wherein the block information acquisition unit divides the color distribution in each block so that a variance value is smaller than a predetermined threshold.   The image coding apparatus according to claim 1, wherein the difference image coding unit performs orthogonal transform using a plurality of blocks divided in a mode indicated by the block information as processing units.   The image coding apparatus according to claim 13, wherein the difference image coding unit performs discrete cosine transform as the orthogonal transform.   The image encoding device according to claim 13, wherein the difference image encoding unit performs discrete wavelet transform as the orthogonal transform.   The image coding apparatus according to any one of claims 1 to 3, wherein the difference image coding unit performs entropy coding using a plurality of blocks divided in a mode indicated by the block information as a processing unit. Output means for outputting encoded data encoded by the differential image encoding means;
The image encoding apparatus according to claim 1, wherein the output unit outputs the encoded data for each block size of an encoding processing unit.   The image encoding apparatus according to claim 17, wherein the output unit outputs the encoded data in order from encoded data having a larger processing unit block size. Encoded data input means for inputting encoded data obtained by encoding an image;
Basic image decoding means for decoding a part of the encoded data to obtain a basic image serving as a base of the original image;
Block information acquisition for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image decoded by the basic image decoding means, and acquiring block information indicating a division mode Means,
Differential image decoding means for decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as processing units to obtain a differential image;
An image decoding apparatus comprising: Encoded data input means for inputting encoded data obtained by encoding an image;
Basic image decoding means for decoding a part of the encoded data to obtain a basic image obtained by down-sampling the original image;
Block information acquisition for dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image decoded by the basic image decoding means, and acquiring block information indicating a division mode Means,
Differential image decoding means for decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as processing units to obtain a differential image;
An image decoding apparatus comprising: Encoded data input means for inputting encoded data obtained by encoding an image composed of a plurality of continuous frames;
A block showing a mode of division by using a previous frame of a frame to be decoded included in the encoded data as a basic image, dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image Block information acquisition means for acquiring information;
Differential image decoding means for decoding the encoded data using a plurality of blocks divided in the mode shown in the block information as processing units to obtain a differential image;
Original image combining means for combining the original image of the frame based on the basic image and the difference image;
An image decoding apparatus comprising:   The image decoding according to any one of claims 19 to 21, wherein the block information acquisition unit divides an edge extracted image obtained by extracting an edge of the basic image so that an edge strength in each block is smaller than a predetermined threshold. Device.   The image decoding device according to claim 22, wherein a plurality of threshold values used in the block information acquisition unit are set in accordance with a block size.   The image decoding apparatus according to claim 22, wherein a threshold value used by the block information acquisition unit is input together with the encoded data. An edge extraction filter decoding means for decoding an edge extraction filter included in the encoded data;
The image decoding device according to any one of claims 19 to 21, wherein the block information acquisition unit extracts an edge of the basic image using the edge extraction filter.   The image according to any one of claims 19 to 21, wherein the block information acquisition unit obtains a color distribution from a color difference component of the basic image, and acquires the block information by dividing into a plurality of blocks based on the color distribution. Decryption device.   27. The image decoding device according to claim 26, wherein the block information acquisition unit divides the block so that a variance value of a color distribution in each block is smaller than a predetermined threshold value.   The image decoding device according to any one of claims 19 to 21, wherein the difference image decoding unit performs orthogonal transform using a plurality of blocks divided in a mode indicated by the block information as a processing unit.   The image decoding apparatus according to claim 28, wherein the difference image decoding means performs discrete cosine transform as the orthogonal transform.   29. The image decoding apparatus according to claim 28, wherein the differential image decoding means performs discrete wavelet transform as the orthogonal transform.   The image decoding device according to any one of claims 19 to 21, wherein the difference image decoding unit performs entropy decoding using a plurality of blocks divided in a mode indicated by the block information as processing units. An original image input step for inputting the original image;
A basic image acquisition step of acquiring a basic image as a basis of the original image;
A difference image calculating step for obtaining a difference image between the original image and the basic image;
A block information acquisition step of dividing the basic image into a plurality of blocks of different sizes based on edge information included in the basic image, and acquiring block information indicating a division mode;
A differential image encoding step for encoding a differential image using a plurality of blocks divided in the mode shown in the block information as processing units;
An image encoding method comprising: An encoded data input step for inputting encoded data obtained by encoding an image;
A basic image decoding step of decoding a part of the encoded data to obtain a basic image serving as a base of an original image;
Block information acquisition step of dividing the basic image into a plurality of blocks having different sizes based on edge information included in the basic image decoded in the basic image decoding step, and acquiring block information indicating a division mode When,
A differential image decoding step of decoding the encoded data using a plurality of blocks divided in a mode shown in the block information as processing units to obtain a differential image;
An image decoding method comprising:

Images