KR101295158B1 - Image codec system for supporting spatial random access and image encoding/decoding equipment and method thereof - Google Patents

Abstract

An image codec system supporting spatial random access, and an image encoding / decoding apparatus and method configuring the same, are described. In an image encoding apparatus of an image codec system according to an embodiment of the present invention, a decoded pixel value of a pixel of an adjacent block, which is used as a prediction value in intra prediction encoding, together with an image bitstream, is separately stored as prediction data. When requested by the user, prediction data is separately extracted and transmitted along with the image bitstream of the target image. The image decoding apparatus of the image codec system according to the present embodiment reconstructs the target image using only the received image bitstream and the prediction data of the target image. According to such an embodiment of the present invention, since the video bitstream does not need to be transmitted to an area not included in the target video, the transmission efficiency can be improved.

Description

Image codec system for supporting spatial random access and image encoding / decoding equipment and method

1A to 1C are diagrams showing a relationship of a transmission area according to the size of a fragment image, respectively.

2 is a diagram illustrating a relationship between a current macroblock and neighboring macroblocks necessary for decoding thereof.

3 is a block diagram illustrating a configuration of an image codec system according to a first embodiment of the present invention.

4 is a diagram showing 16 4x4 blocks in a current macroblock and neighboring 4x4 blocks for which mode information is required for decoding the 4x4 blocks.

FIG. 5 is a diagram illustrating types classified according to positions of respective macroblocks in a target image.

6 is a block diagram schematically illustrating a process of obtaining mismatch data (M-Data) according to an embodiment of the present invention.

7 is a block diagram illustrating a configuration of an image codec system according to a second embodiment of the present invention.

The present invention relates to an image codec, and more particularly, to an image codec system supporting spatial random access, and an image encoding / decoding apparatus and method for configuring the same.

Recently, with the digitization of broadcasting, convergence of broadcasting and communication has been actively progressed. Accordingly, the broadcasting environment is also changing to a complex form that allows two-way communication in the existing one-way reception and interworking heterogeneous networks and accommodates various terminals. A representative example of this trend is ubiquitous television (TV). Ubiquitous TV is a TV that can be watched anytime and anywhere when digital devices are connected to the Internet. In order to watch ubiquitous TV, a ubiquitous TV tuner can be connected to an antenna of a cable or terrestrial TV and connected to a high speed internet line. With such ubiquitous TV, the ubiquitous TV tuner can be remotely controlled using a portable terminal such as a laptop, a personal digital assistant (PDA), a WiBro terminal, or a portable multimedia player (PMP). In addition, various media such as movies, photos, and animations can be viewed. The ubiquitous TV service is rapidly emerging as a next-generation service in that a user can obtain desired information anytime, anywhere, or view a video using a terminal of a user's desired form.

  However, such a ubiquitous environment that can share information in various terminals is creating a new problem of resolution mismatch between terminals. For example, the recent high-definition television (HDTV) supports up to 1920 × 1080 resolution, and the new film-only digital format in the US-based film industry is based on 4K x 2K digital cinema video. have. On the other hand, PDAs, portable multimedia players (PMPs), and WiBro terminals only support resolutions up to Common Intermediate Format (CIF) or Quad Common Intermediate Format (QCIF).

As such, when the resolutions of the terminals are different from each other, a problem may occur in that one video data cannot be used interchangeably among several terminals. And this problem is an obstacle to realizing a ubiquitous environment such as ubiquitous TV service.

One way to solve the problem of resolution mismatch described above is to down-sample a high resolution image transmitted to a TV or the like, and then make a low resolution image and transmit it to a portable terminal. However, providing a downsampled low resolution image cannot satisfy a user's desire to view a high quality screen even through a portable terminal. In particular, a user watching a music video, a sports game such as soccer, and a home shopping may sometimes want to watch a specific part of an image in detail, and the downsampling method is more suitable to satisfy the needs of these users. it's difficult. However, transmitting the entire high resolution image to the portable terminal not only wastes limited transmission channel resources, but also causes a problem that the portable terminal needs to perform decoding on an unnecessary portion of the image.

One method proposed to solve this problem is spatial random access (Spatial Random Access). Spatial random access is to search and transmit and restore only a part of an image in an arbitrary position of the image. When using this method, a specific part considering the user's intention and the size of the display window (not data about the entire image) can be used. It is possible to perform partial decoding for transmitting and decoding only the minimum data necessary for displaying the target image). Therefore, by using an image codec system that supports spatial random access, it may be possible to view a high-definition image such as an HDTV using a portable terminal.

Due to the diversification and popularization of portable terminals, the necessity of an image codec system supporting such a random access is increasing. At present, one of the subgroups of MPEG is the Free View-point TV (F-TV) and the 3D TV (3D TV) in relation to Multi-view Video Coding (MVC). Standardization is underway as a Target Application. As a result of the 76th MPEG Swiss Meeting held in April 2006, only part of the image is accessible. View Scalability is a requirement. Free view scalability is a technique that enables decoding by separating only a part of an image acquired through each camera in multiview video coding.

In the case of panoramic images as well as 3D images, such spatial random access is considered to be essential. In general, since a panorama image is a reconstructed image of images taken at various viewpoints into a single image, the size of the image is more than tens of times that of the general image. It is practically impossible to transmit and decode the entire panoramic image considering the limitations of the transmission channel and the processing power of the decoder. Therefore, in the case of panoramic video, if only a part of the video can be transmitted and decoded using an image codec system that supports spatial random access, it not only satisfies a user's desire to watch high-definition video for a specific part, but also in terms of transmission. It can be seen that it is quite efficient.

However, it is difficult to efficiently realize such spatial random access with the image codec system developed or proposed to date, that is, the image encoding / decoding apparatus and method. Since the necessity of broadcasting or communication service in which various display devices having different resolutions coexist so far has not been high, there has been no extensive research on a concrete method for realizing such a random access. In recent years, however, research on this topic has been slowly progressing. A representative article, "Efficient Representation and Interactive Streaming" proposed by Carsten Grnheit et al. (Proc. ICIP 2002, IEEE Internation Conference on Image Processing, Rochester, NY, USA, September 22) -25, 2002).

In this paper, we propose an image codec system that encodes, transmits, and decodes an intra mode whole image into a plurality of fragment images, and encodes each fragment image unit. The image codec system proposed in the above paper includes the target image, not all of the data for the entire image, in consideration of the size of the display and the position and size of the image (target image) of the region that the user wants to view in the entire image. Only the data of the fragment image is transmitted and decoded.

1A to 1C are diagrams showing the relationship between the size of a fragment image partitioning an entire image and an area to be transmitted to a decoding apparatus in order to display a target image in the case of the method presented in the paper. have. Here, the “target image” refers to a partial image corresponding to a display window size that the user wants to watch in the entire image.

1A to 1C, it can be seen that when the size of the target image is constant, as the unit size of the fragment image is smaller, the area to be transmitted to the decoding apparatus is relatively smaller. For example, as shown in FIG. 1C, when the size of the fragment image is made very small, the size of the target image and the size of the transmission area are almost similar. When the area to be transmitted to the decoding apparatus is reduced in this way, the transmission of data to the area not included in the target video can be reduced in proportion to the area, so that the transmission efficiency can be improved by that amount.

However, general image encoding techniques perform intra prediction encoding on intra mode images. This is because the intra-prediction encoding uses the pixel value of the adjacent block to encode the compression ratio compared to the encoding without the prediction. Therefore, splitting the entire screen into a plurality of fragment images and performing encoding independently for each fragment image unit is contrary to the purpose of performing such intra prediction encoding. As a result, in the case of an intra image performing intra prediction encoding, it is not preferable to reduce the unit size of the fragment image to a predetermined size or less (for example, the size as shown in FIG. 1C) in consideration of coding efficiency.

As described above, an image encoding method that supports an existing spatial random access method that divides an entire image into fragment images and uniformly encodes all fragment images, reduces the size of the fragment image to reduce the size of the fragment image. And the decoding efficiency can be improved. However, reducing the size of the fragment image is problematic in that it reduces the compression efficiency. On the contrary, increasing the size of the fragment image is contrary to the purpose of encoding the fragment image unit, and causes a problem of transmitting and decoding an unnecessary region.

SUMMARY OF THE INVENTION The present invention has been made to solve these problems, and the technical problem to be achieved by the present invention is to efficiently use a limited transmission channel, regardless of the type of display device, the user wants for 3D, panoramic or general video The present invention provides an image codec system that supports spatial random access to view a high resolution image of a specific portion, an image encoding server, an image decoding apparatus, and a method thereof.

An embodiment of the present invention for achieving the above technical problem is an image encoding apparatus that supports a spatial random access to the untitled target image of the entire image, the image bitstream by performing intra prediction encoding on the entire image An image coding unit for generating an image, a prediction data memory for storing prediction data (P-Data) used for generating a prediction block for a current block of the entire image in an intra prediction encoding process, and the whole And an extractor for extracting the image bitstream and the prediction data for the target image from the image bitstream for the image and the prediction data stored in the prediction data memory.

According to an aspect of the embodiment, the prediction data may include a decoded pixel value of a pixel adjacent to the current block. The prediction data memory may also store prediction mode data of a block adjacent to the current block and / or number of non-zero coefficients (NNZ) data of a block adjacent to the current block.

According to another aspect of the embodiment, the extractor extracts an image bitstream for the target image from the image bitstream of the entire image and the prediction data for the entire image from the image bitstream extractor The predictive data extractor may include a predictive data extractor for extracting predictive data. In this case, the predictive data extractor may extract a predictive data extractor for the uppermost, leftmost, and / or rightmost macroblock of the target image. Predictive data can be extracted.

Another embodiment of the present invention for achieving the above technical problem is an image decoding apparatus that supports the spatial random access to the target image that is part of the entire image, the image bitstream and the uppermost side of the target image, And an image decoding unit for reconstructing the target image by performing intra prediction prediction decoding using prediction data (P-Data) for the leftmost and / or rightmost block.

According to an aspect of the embodiment, it may further comprise a prediction data memory for storing the prediction data sent to the image decoding unit.

An image codec system according to an embodiment of the present invention for achieving the above technical problem includes an image encoding apparatus and an image decoding apparatus according to the embodiment of the present invention described above.

Another embodiment of the present invention for achieving the above technical problem is an image encoding apparatus that supports the spatial random access to the target image that is part of the entire image, the encoding independently of the entire image in a block unit of a predetermined size A first image encoding unit for performing a first image bitstream by performing an intra-prediction encoding on the entire image to generate a second image bitstream, and the first image encoding unit Mismatch data, M−, which is a difference value between the decoded first pixel value of the first image bitstream generated by the decoded second pixel value of the second image bitstream provided by the second image coding unit. Inconsistent data memory for acquiring and storing Data in blocks, and the top, leftmost, and / or rightmost sides of the target image. A video bitstream is extracted from the first video bitstream for the block, and a bitstream from the second video bitstream for the blocks other than the top, left, and rightmost blocks among the blocks of the target video. And a mismatch data extractor for extracting the mismatch data for the uppermost, leftmost, and / or rightmost blocks of the target image.

According to an aspect of the embodiment, the difference value for the pixels located in the lowermost column and the rightmost row of each block may be stored in the mismatched data memory. The mismatch data memory may further store prediction mode data of adjacent blocks and / or number of non-zero coefficients (NNZ) data of adjacent blocks.

Another embodiment of the present invention for achieving the above technical problem is an image decoding apparatus that supports the spatial random access to the target image that is part of the entire image, the first region of the target image is independent of the block unit of a predetermined size Decoding is performed, and an intra prediction prediction decoding is performed on a second area of the target image in units of blocks of a predetermined size, and a decoded pixel value of an adjacent block used for intra prediction prediction decoding is applied to the second area. In the case of a pixel belonging to one region, an image decoding unit which performs intra prediction prediction decoding after adding mismatched data to the independent decoded pixel value is included.

According to an aspect of the embodiment, the first area includes blocks located at the top, leftmost, and / or rightmost sides of the target image. The pixel value for adding the mismatched data may be pixel values of pixels located in the rightmost row and / or the lowest column in each block belonging to the first region.

An image codec system according to another embodiment of the present invention for achieving the above technical problem includes an image encoding apparatus and an image decoding apparatus according to the embodiment of the present invention described above.

Another embodiment of the present invention for achieving the above technical problem is an image encoding method that supports spatial random access to a target image that is part of an entire image, and performs intra prediction encoding on the entire image. Generating an image bitstream and storing prediction data (P-Data) used to generate a prediction block in the process of performing the intra prediction encoding, and from the user to the target image. And extracting and transmitting the image bitstream and the prediction data of the target image from the image bitstream and the prediction data, respectively, when there is a request for transmission.

Another embodiment of the present invention for achieving the above technical problem is an image decoding method for supporting a spatial random access to the target image that is part of the entire image, the step of receiving prediction data with the image bitstream in the target image Generating a prediction block using the prediction data, and reconstructing the target image using the image bitstream and the prediction block.

Another embodiment of the present invention for achieving the above technical problem is an image encoding method that supports the spatial random access to the target image that is part of the entire image, the encoding independently of the entire image in a block unit of a predetermined size Generating a first image bitstream by performing intra-prediction encoding on the entire image to generate a second image bitstream, and decoding the first pixel value of the first image bitstream and the first image bitstream. Obtaining and storing mismatch data (M-Data), which is a difference value of the decoded second pixel value of the second image bit stream, in units of blocks, and when the user requests a transmission of the target image, Among the image bitstream, the second image bitstream, and the mismatched data, the topmost, leftmost, and / or rightmost of the target image. An image bitstream is extracted from the first image bitstream for a block, and image bits are extracted from the second image bitstream for blocks other than the uppermost, leftmost, and rightmost blocks among the blocks of the target image. Extracting a stream, and extracting and transmitting the mismatched data for the uppermost, leftmost, and / or rightmost blocks of the target video.

Another embodiment of the present invention for achieving the above technical problem is an image decoding method for supporting a spatial random access to the target image that is part of the entire image, the first image bitstream, the second image bitstream to the target image And restoring pixel values for the first region of the target image by decoding the first image bitstream, and pixel values for the restored first region using the mismatch data. Reconstructing the pixel value of the second region of the target image by performing intra prediction prediction decoding using the corrected pixel value of the first region and the second image bitstream. do.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Since the embodiments described below are for the purpose of illustrating the technical idea of the present invention, the technical idea of the present invention should not be construed as being limited by the embodiments. In the description of the following embodiments, the names of each component may be referred to as other names in the art, and if they have functional similarity and identity, they may be regarded as equivalent to the embodiments of the present invention even though other names are used. have. Similarly, even if an embodiment in which the configuration on the drawings is partially modified is adopted, both can be regarded as an equivalent configuration if there is functional similarity and identity. Reference numerals added to the respective components in the description of the embodiment and the drawings are merely described for convenience of description.

An image codec system supporting spatial random access according to the present invention, an image encoding server and an image decoding apparatus constituting the same, and a method thereof are an apparatus or method for encoding or decoding an intra mode image. In order to achieve the above technical problem, the present invention transmits prediction data (P-Data) or mismatch data (M-Data) to the image decoding apparatus separately from the image bitstream. Hereinafter, each of these will be described in detail.

First Embodiment

The video codec system according to the first embodiment of the present invention uses P-Data. 'P-Data' refers to the decoded pixel values of adjacent pixels used to predict the pixel values of the current block during intra prediction encoding. In the case of sequentially encoding / decoding each macroblock according to a raster scan order, the P-Data is provided from macroblocks located to the left and / or above the current macroblock.

FIG. 2 is a block diagram illustrating neighboring macroblocks that provide P-Data required for decoding a current macroblock when decoding according to a raster scan order. Referring to FIG. 2, in order to decode a current macroblock, decoded pixel values of neighboring macroblocks left and up adjacent to the current macroblock are required. In order to decode the neighboring macroblock, the decoded pixel values of neighboring macroblocks adjacent to the left side of the macroblock and neighboring macroblocks upwardly are required. As a result, in order to decode the current macroblock, decoding of both neighboring macroblocks located on the left side and neighboring macroblocks located on the upper side should be preceded.

However, transmitting and decoding the video bitstreams of all the blocks shown in FIG. 2 to the video decoding apparatus for decoding the current macroblock need not only reduce the transmission efficiency but also display the excessive data transmission. There is a problem that decoding should be performed even for a missing image bitstream. This problem occurs not only when the entire image is regarded as an intra mode image and the encoding / decoding is performed, but also when the entire image is divided into a plurality of fragment images and the encoding / decoding is performed in units of fragment images.

In this embodiment, in order to solve these problems, P-Data and the like are separately stored during encoding, and only the P-Data and the like necessary for reconstructing the image bitstream for the target image that the user wants to display are extracted and decoded. To the device. In addition, the image decoding apparatus performs decoding by using the image bitstream and the received P-Data of the received target image.

3 is a block diagram illustrating a configuration of an image codec system 100 according to an embodiment of the present invention. Referring to FIG. 3, the image codec system 100 includes an image encoding apparatus 110 and an image decoding apparatus 120. The image encoding apparatus 110 includes an image encoding unit 112, a first P-Data memory 114, an image bitstream extractor 116, and a P-Data extractor 118. The image decoding apparatus 120 includes an image decoding unit 122 and a second P-Data memory 124.

First, the video encoding apparatus 110 will be described.

The image encoding unit 112 of the image encoding apparatus 110 is a means for generating an image bitstream by compressing the entire input image using a predetermined encoding method. The image encoding unit 112 is preferably an encoder conforming to a standard for image compression such as H.264 / AVC or VC-1, but is not limited thereto. The generated image bitstream may be stored in a predetermined storage means or in a buffer memory or the like in the case of real time transmission.

The image encoding unit 112 according to the present embodiment is an encoder that supports intra prediction encoding on an intra mode image, and encodes an input image in units of blocks having a predetermined size. For example, the image encoding unit 112 may perform intra prediction encoding on a macroblock basis for an intra mode image, but is not limited thereto. For example, the image encoding unit 112 may perform encoding in units of macroblocks, 4 × 4 blocks, or 8 × 8 blocks, or adaptively change the size of the unit block.

The first P-Data memory 114 is a means for storing P-Data, i.e., decoded pixel values of neighboring pixels used to predict pixel values of the current macroblock at the time of encoding in the image encoding unit 112. According to an exemplary embodiment, the first P-Data memory 114 also stores metadata information of neighboring blocks necessary for intra prediction encoding the current macroblock. To this end, the image encoding unit 112 sends the P-Data generated in the encoding process to the first P-Data memory 114. The first P-Data memory 114 may be integrated with or separated from other components of the image encoding apparatus 110, for example, a frame memory or an image bitstream storage memory included in the image encoding unit 112. May form an independent storage unit.

The type and / or specific value of the P-Data stored in the first P-Data memory 114 may vary according to the intra prediction encoding method supported by the image encoding unit 112. For example, when the image encoding unit 112 is an encoder conforming to the H.264 / AVC standard, the P-Data stored in the first P-Data memory 114 is a pixel adjacent to the current macroblock. Is a reconstructed pixel value (reconstructed pixel data). The first P-Data memory 114 may include information (prediction mode data) indicating a prediction mode of a 4 × 4 block adjacent to the current block in the 4 × 4 block mode and / or a 4 × 4 block in the current block. In the case of the mode, information (NNZ data) indicating the number of non-zero coefficients (NNZ) of 4x4 blocks adjacent thereto may be stored, which will be described in detail later.

The image bitstream extractor 116 extracts only the bitstream of the target image desired by the user from among the bitstreams of the entire image. That is, when a user requests data by specifying a position (target image) of a portion that the user wants to watch among the entire images, the image bitstream extractor 116 generates an image bitstream for macroblocks constituting the target image. Extract.

The P-Data extractor 118 performs a function of extracting P-Data or the like for a target image desired by a user from among data stored in the first P-Data memory 114. As described above, the P-Data may be decoded pixel values of pixels adjacent to the left and / or upward of the macroblock constituting the target image, and the decoded pixel values are data for each macroblock as the first data. It may be stored in the P-Data memory 114. According to the present embodiment, the P-Data extractor 118 does not need to extract the P-Data for the entire macroblock constituting the target image, and is located at the leftmost, rightmost and topmost positions of the target image. It is enough to extract only the P-Data for the macroblock. Details thereof will be described later along with a detailed description of the P-Data.

Subsequently, the video decoding apparatus 120 will be described.

The image decoding unit 122 of the image decoding apparatus 120 is a means for reconstructing a target image that the user wants to watch. The image decoding unit 122 may include an image bitstream (ie, a bitstream extracted by the image bitstream extractor 116) for the target image and a P-Data (ie, P-Data extractor 118) for the target image. The target image is reconstructed using the extracted P-Data).

The second P-Data memory 124 of the image decoding apparatus 120 is a means for receiving and storing P-Data or the like used to restore the target image. The second P-Data memory 124 may be integrated with other components of the image decoding apparatus 120, for example, a memory included in the image decoding unit 122, or may be separated from the image decoding unit 122 and the like. It may form a unit.

Subsequently, when the image encoding unit 112 and the image decoding unit 122 according to the first embodiment described above are encoders or decoders conforming to the H.264 / AVC standard, respectively, the first P-Data memory. Types of data stored in 114 and data extracted by the P-Data extractor 118 for transmission to the image decoding apparatus 120 when the user requests data for a predetermined target screen will be described in detail. . In the case of using an encoder and a decoder conforming to the H.264 / AVC standard, the first P-Data memory 114 stores decoded pixel values (P-Data) of pixels adjacent to the current macroblock, and the current block In this 4x4 block mode, prediction mode data of a neighboring 4x4 block adjacent to this, and NNZ data of a neighboring 4x4 block adjacent thereto when the current block is a 4x4 block may also be stored.

(1) reconstructed pixel data

Intra Prediction, defined in the H.264 / AVC standard, has 16x16 block mode and 4x4 block mode for luminance components, and 8x8 block mode for chrominance components. exist. Only reconstructed pixel data for the luminance component will be described below.

In the case of the 4x4 block mode, there are nine prediction modes (prediction mode 0 to prediction mode 8). In order to apply the 4x4 block mode, the macroblock is first divided into 16 4x4 blocks. For each 4x4 block, encoding is performed by selecting an optimal prediction mode from the nine prediction modes. The nine prediction modes are Vertical (prediction mode 0), Horizontal (prediction mode 1), DC (prediction mode 2), Diagonal down-left (prediction mode 3), Diagonal down-right (prediction mode 4), Vertical-right (Prediction mode 5), Horizontal down (prediction mode 6), Vertical left (prediction mode 7), and Horizontal-up (prediction mode 8). Applying each of the nine prediction modes to one macroblock consisting of 16 4 × 4 blocks, in order to generate one predicted luminance macroblock, a maximum of 37, i.e., the top 21 neighboring pixels and the left 16 neighbors are generated. The decoded luminance value of the pixel is required.

In case of 16 × 16 block mode, four prediction modes exist. The four prediction modes are Vertical (prediction mode 0), Horizontal (prediction mode 1), DC (prediction mode 2), and Plane (prediction mode 3). When each of the four prediction modes is applied to a macroblock, the decoded luminance values of up to 33, i.e., 17, upper and 16 adjacent pixels are used to generate one predicted luminance macroblock.

Therefore, the decoded luminance value of up to 37 pixels adjacent to each macroblock may exist as P-Data, and whether or not the reconstructed pixel data of several pixels is actually used as a prediction value in predicting the macroblock. It may vary depending on the block mode of each macroblock and / or the prediction mode of the unit block constituting each macroblock.

(2) prediction mode data

In H.264 / AVC, the prediction mode information for the current 4x4 block is adaptively applied in order to reduce the number of bits for the information (prediction mode information) indicating the prediction mode in the intra 4x4 block mode. Assign. More specifically, when a small mode is selected as a reference mode from among prediction modes of 4 × 4 blocks located on the left and top of the current 4 × 4 block, and the selected reference mode and the prediction mode of the current 4 × 4 block are the same. Indicates that they are the same by transmitting only one bit. If the prediction mode of the current 4x4 block and the reference mode are not the same, 1-bit information indicating that the prediction mode is different and the remaining 8 prediction modes except for the reference mode are displayed in 3 bits so as to display the current 4x4 block. Information indicating the prediction mode is displayed as 4 bits. The allocation of the adaptive prediction mode information can improve coding efficiency by adaptively expressing nine prediction modes in one bit or four bits as compared with collectively representing four bits.

FIG. 4 is a diagram for showing a 4x4 block requiring prediction mode information of neighboring 4x4 blocks among 16 4x4 blocks (Nos. 0 to 15) within a current macroblock. Referring to FIG. 4, the 4x4 block '0' requires prediction mode information of the left 4x4 block (0A) and the upper 4x4 block (0B), and the '1', '4', and ' 4x4 block 5 'requires prediction mode information of the upper 4x4 blocks 1B, 2B, and 3B, and 4x4 blocks' 2', '8', and '10' are left 4x4. The prediction mode information of the blocks 1A, 2A, and 3A is required.

Therefore, in order to recover one macroblock, prediction mode information of up to eight neighboring 4x4 blocks (Nos. 0, 1, 2, 4, 5, 8, and 10) is required. Together with the video decoding apparatus.

(3) NNZ data

H.264 / AVC uses Context Adaptive Variable Length Coding (CAVLC) for entropy coding. CAVLC performs entropy encoding by adaptively selecting one of several VLC tables according to the situation of neighboring blocks. The number of neighboring nonzero coefficients NNZ is used to determine the VLC table of the current 4x4 block. More specifically, the VLC table to be applied to the encoding of the current 4x4 block by obtaining the average value (denoted in nC) of the NNZ values of the 4x4 blocks located on the left and top of the current 4x4 block and using the average value Determine.

Therefore, even in the case of NNZ information of adjacent 4x4 blocks, similar to the prediction mode information of the adjacent 4x4 blocks described above, up to eight peripheral 4x4 blocks (0, 1, 2) in order to recover one macroblock. , Nos. 4, 5, 8 and 10) are required, and these NNZ information should also be transmitted to the video decoding apparatus together with the P-Data.

The three types of data described above, namely P-Data, prediction mode information, and NNZ information, are sent from the image coding unit 112 to the first P-Data memory 114 for each macroblock at the time of encoding and stored. . For example, the P-Data may be stored in the first P-Data memory 114 as additional information of the corresponding macroblock. The P-Data extractor 118 considers the location and / or block mode of each macroblock in the target image, and among the data for each macroblock stored in the first P-Data memory 114. All or part of the data is extracted and transmitted to the image decoding apparatus 120.

FIG. 5 is a diagram for describing in detail a type of data transmitted by the P-Data extractor 118 to the image decoding apparatus 120 according to a display window, that is, a macroblock position and / or a block mode in a target image. . According to the present exemplary embodiment, one or more pieces of data of the three types of data are likely to be transmitted to the macroblocks positioned at the left and right edges and the upper edge of the target image, that is, the macroblocks ①, ②, ③ and ⑦. For the remaining macroblocks, there is no need to separately transmit P-Data. Hereinafter, the types of data to be transmitted in the case of the macroblocks ①, ②, ③ and ⑦ will be described in more detail with reference to examples.

An example of this embodiment is a case where all three types of data are extracted and transmitted for all macroblocks corresponding to macroblocks ①, ②, ③, and ⑦. This case has the advantage of simplifying the configuration and operation of the P-Data extractor 118 in that it uniformly processes all macroblocks requiring transmission of P-Data and the like. However, in this case, there may be a certain limit in maximizing the transmission efficiency in that unnecessary data transmission may occur in decoding the macroblock.

Another example of the present embodiment is a case where only data necessary for decoding the macroblock is transmitted in consideration of the block mode and the prediction mode of the macroblocks ①, ②, ③ and ⑦. More specifically, all three types of data described above are transmitted for the macroblock ①, and reconstructed pixel data for the left 16 pixels for each macroblock among the three types of data described above for the macroblock ②, and the left side. Prediction mode data for four 4x4 blocks and / or NNZ data for the left four 4x4 blocks may not be transmitted. For the macroblock ③, the reconstructed pixel data of the top 16 pixels for each macroblock, the prediction mode data for the top 4 4 × 4 blocks, and / or the top 4 4 × 4 of the above three types of data. NNZ data for the block may not be transmitted. In the case of the macroblock ⑦, only reconstructed pixel data of four pixels positioned adjacent to each of the macroblocks in the right and upper directions can be transmitted.

According to another aspect of the present embodiment, it is possible to selectively apply the contents in the following cases, based on the transmission of all three data described above for the macroblocks ①, ②, ③.

When the macroblock ② is in the 16 × 16 block mode and the horizontal prediction mode, reconstructed pixel data is not transmitted.

When the macroblock ③ is in the 16 × 16 block mode and the vertical prediction mode, reconstructed pixel data is not transmitted.

In the case of the macroblock ②, the reconstructed pixel data of the 16 pixels adjacent to the left side and the NNZ information of the 4 4x4 blocks on the left side are not transmitted.

In the case of the macroblock ③, the reconstructed pixel data of the 16 adjacent pixels upward and the NNZ information of the 4 upper 4 × 4 blocks are not transmitted.

In the case of macroblock ⑦, only reconstructed pixel data for four pixels adjacent to the upper right side is transmitted.

The prediction mode information of adjacent 4x4 blocks is transmitted only when both the current block mode and the neighboring block modes are 4x4 block modes.

According to the image codec system according to the first embodiment of the present invention described in detail above, the image encoding server performs the intra prediction encoding on the entire image in the intra mode to generate the image bitstream in the same manner as the conventional method. In addition, the prediction pixel data used for the intra prediction encoding is stored in a separate memory. When the user requests data for a target video designating a specific part of the entire video, the video encoding server, together with the video bitstream for the target video, provides P-Data or the like necessary for decoding the video bitstream. Transmission to the video decoding apparatus. Also, the image decoding apparatus restores the requested target image by using the received image bitstream and P-Data.

According to the video codec system according to the first embodiment of the present invention, transmission efficiency can be improved because only the video bitstream and the P-Data for the target video are transmitted instead of the entire video. In addition, the image decoding apparatus does not need to perform decoding on an unnecessary image portion (that is, a portion of the entire image that does not correspond to the target image). As a result, since the image codec system according to the first embodiment of the present invention supports spatial random access and efficiently uses a limited transmission channel, a user can watch a high quality image even for a target image which is a part of the entire image. It is possible.

Second Embodiment

The second embodiment of the present invention is a case of using mismatched data (M-Data). In general, 'M-Data' refers to reconstructed pixel data obtained by encoding and then decoding each macroblock by an intra prediction encoding method, and reconstructed pixel data obtained by separately encoding and then decoding each macroblock without performing intra prediction encoding. The difference value of. In the present specification, the term "M-Data" may be used to mean a difference value for a part of the macroblock, not a difference value for the entire macroblock. For example, as illustrated in FIG. 6, the term M-Data may be used to indicate a difference value between a lower left column and a left and right row of a macroblock.

7 is a block diagram illustrating a configuration of an image codec system 200 according to a second embodiment of the present invention. Referring to FIG. 7, the image codec system 200 includes an image encoding server 210 and an image decoding apparatus 220. The image encoding server 210 may include a first image encoding unit 211, a second image encoding unit 212, a first M-Data memory 214, an image bitstream extractor 216, and an M-Data extractor ( 218). The image decoding apparatus 220 includes an image decoding unit 222 and a second M-Data memory 224.

First, the video encoding server 210 will be described. The image encoding server 210 according to the present embodiment includes two types of image encoding units 211 and 212, a first M-Data memory 214, and an M-Data extractor 218. There is a difference from the image encoding server 110 according to the first embodiment.

Means for generating a first video bitstream and a second video bitstream, respectively, by compressing the entire video inputted by the first video encoding unit 211 and the second video encoding unit 212 using a predetermined encoding method. to be. The first and second image encoding units 211 and 212 are preferably encoders that conform to a standard for image compression such as H.264 / AVC or VC-1, but are not limited thereto. The video bitstreams generated from the first video encoding unit 211 and the second video encoding unit 212 may be stored in predetermined storage means or in a buffer memory in the case of real time transmission.

According to the present exemplary embodiment, the first image encoding unit 211 is an encoder that independently encodes an intra mode image in units of predetermined sizes, for example, macroblocks, to generate a first image bitstream. That is, the first image encoding unit 211 does not perform intra prediction encoding that is encoded by using decoded pixel data of an adjacent block. On the other hand, the second image encoding unit 212 is an encoder that performs intra prediction encoding on an intra mode image to generate a second image bitstream. Therefore, the second image encoding unit 212 is the same as the image encoding unit 112 of the first embodiment described above.

The first M-Data memory 214 is generated by the decoded pixel value (first decoded pixel value) for each macroblock generated by the first image encoding unit 211 and by the second image encoding unit 212. Means for storing the difference value (differential pixel data) of the decoded pixel value (second decoded pixel value) for each macroblock. In order to obtain such a difference value, a subtractor 215 may be further provided at an input side of the first M-Data memory 214. The first M-Data memory 214 stores difference data for all pixels (256 pixels) in units of macroblocks, or as shown in FIG. 6, the rightmost 16th row and the lowest 16th block of each block. Only difference data for pixels located in the column, that is, 31 pixels in total, can be stored, with the latter being more preferred. The first M-Data memory 214 may also store some of the data generated by the second image coding unit 212, more specifically, prediction mode data and NNZ data.

The image bitstream extractor 216 extracts only a bitstream of a target image desired by a user from among a bitstream of the entire image, that is, a first image bitstream and a second image bitstream. More specifically, according to the present exemplary embodiment, the image bitstream extractor 216 may include macroblocks positioned at both edges and macroblocks positioned at the top of macroblocks constituting the target image (ie, FIG. 7 described above). Blocks corresponding to the macroblocks 1, 2, and 3) are extracted from the first video bitstream. On the other hand, data is extracted from the second image bitstream for the remaining macroblocks that do not use the first image bitstream among the macroblocks constituting the target image.

The M-Data extractor 218 extracts differential pixel data, prediction mode data, and / or NNZ data, which are M-Data for the corresponding macroblock, from the M-Data stored in the first M-Data memory 214. It performs the function. More specifically, the M-Data extractor 218 may include macroblocks positioned at both edges of the macroblocks constituting the target image and macroblocks positioned at the uppermost side (ie, macroblocks ① and ② of FIG. 5 described above). Difference pixel data) is extracted. The prediction mode data and the NNZ data are extracted for the remaining macroblocks.

Subsequently, the video decoding apparatus 220 will be described.

The image decoding unit 222 of the image decoding apparatus 220 is a means for reconstructing a target image that the user wants to watch. The image decoding unit 222 may be located at the top of the macroblocks located at both edges of the macroblocks constituting the target image by using the received first image bitstream and differential pixel data (M-Data). The macro blocks (that is, the blocks corresponding to the macro blocks ①, ②, and ③ of FIG. 5 described above) are restored. The remaining macroblocks not corresponding to macroblocks ①, ②, and ③ of the target image are decoded using the received second image bitstream, prediction mode data, and / or NNZ data.

However, the first image bitstream is composed of data encoded independently in macroblock units, and the second image bitstream is composed of data encoded by intra prediction encoding in macroblock units. However, in the case of the macroblocks ④, ⑤, and ⑥ of FIG. 5, the pixel values of the macroblocks ①, ②, and ③ are used as prediction values. Since the pixel values of (1), (2), and (3) are reconstructed, there is a difference from the pixel values of macroblocks (1), (2), and (3) used as prediction values in the video encoding unit 212. Therefore, in the decoding method according to the present embodiment, before performing decoding using the second image bitstream, macroblocks ①, ②, using differential pixel data stored in the second M-Data memory 224. The pixel value corresponding to?, More specifically, the pixel value corresponding to the sixteenth row and / or the sixteenth column in each macroblock is compensated for.

The second M-Data memory 224 of the image decoding apparatus 220 is a means for storing differential pixel data (M-Data), prediction mode data and / or NNZ data, etc. used for reconstruction of the target image. The second M-Data memory 224 may be integrated with other components of the image decoding apparatus 220, for example, a memory included in the image decoding unit 222, or may be separated from and independent of the image decoding unit 222. The phosphorus unit may be formed.

Next, when the image codec system according to the present embodiment is an image codec system conforming to the H.264 / AVC standard, it is stored in the first M-Data memory 214 and transmitted to the image decoding apparatus 120. The above-described data will be described in detail. As described above, the first M-Data memory 214 includes decoded pixel values generated by the first image coding unit 211 for 35 pixels corresponding to the lowermost column and the uppermost row of each macroblock. Prediction mode information (prediction mode data) of M-Data, which is a difference value (differential pixel data) between the decoded pixel values generated by the second image encoding unit 212, and neighboring 4x4 blocks adjacent to the current macroblock, and / Or NNZ information (NNZ data) of a neighboring 4x4 block adjacent to the current macroblock.

(1) differential pixel data

The M-Data memory 214 stores difference values for 31 pixels located in the rightmost row (16th row) and the lowest column (16th column) for each macroblock as differential pixel data. The differential pixel data transmitted to the image decoding apparatus 220 may vary according to positions occupied by each macroblock in the target image. For example, in the case of the macroblock ① of FIG. 5, all differential pixel data for the 31 pixels is transmitted. In the case of macroblock ② of FIG. 5, only differential pixel data for 16 pixels located in the lowermost column is transmitted. In the case of macroblock ③ of FIG. 5, differential pixel data for 16 pixels located in the rightmost row is transmitted. Can only transmit. However, in the following two cases, differential pixel data for some macroblocks may not be transmitted.

Differential pixel data for 16 pixels located in the lowermost column of the macroblock ② when the block corresponding to the macroblock? Of FIG. 5 is in the 16 × 16 block mode and is in the horizontal prediction mode.

Differential pixel data for the 16 pixels located in the rightmost row of the macroblock ③ when the block corresponding to the macroblock ⑥ in FIG. 5 is the 16x16 block mode and is in the vertical prediction mode.

(2) prediction mode data

When the current macroblock is in the 4x4 block mode, prediction mode information of 4x4 blocks located at the rightmost and / or bottommost side of FIG. 4 is stored in the M-Data memory 214 for each macroblock. . Referring to FIG. 4, prediction modes of seven 4 × 4 blocks corresponding to macroblocks 5, 7, 10, 11, 13, 14, and 15 are stored.

The prediction mode data extracted by the M-Data extractor 218 and transmitted to the image decoding apparatus 220 may vary according to the position of the current macroblock in the target image. For example, when the current macroblock corresponds to macroblock ① of FIG. 5, all of the prediction modes of the seven 4 × 4 blocks are transmitted. When the current macroblock corresponds to the macroblock ② of FIG. 5, the prediction modes of the 4 × 4 blocks 10, 11, 14, and 15 of the seven prediction modes are transmitted, and the current macroblock is the macro of FIG. 5. In case of the block ③, the prediction modes of blocks 4, 4, 5, 7, 13 and 15 of the seven prediction modes are transmitted. However, when the macroblock corresponding to macroblock⑤ of FIG. 5 is the 16 × 16 block mode, the prediction mode of the macroblock corresponding to ② of FIG. 5 and the macroblock corresponding to the macroblock⑥ of FIG. 5 are 16 × 16. In the block mode, the prediction mode of the macroblock corresponding to 3 in FIG. 5 may not be transmitted.

(3) NNZ data

When the current macroblock is in the 4x4 block mode, the NNZ data of the 4x4 block located at the rightmost and / or bottommost side of FIG. 4 is stored in the M-Data memory 214 for each macroblock. Referring to FIG. 4, NNZ data of seven 4 × 4 blocks corresponding to macroblocks 5, 7, 10, 11, 13, 14, and 15 is stored.

The NNZ data extracted by the M-Data extractor 218 and transmitted to the image decoding apparatus 220 may vary according to the position of the current macroblock in the target image. For example, when the current macroblock corresponds to macroblock ① of FIG. 5, all of the NNZ data of the seven 4 × 4 blocks are transmitted. When the current macroblock corresponds to macroblock ② of FIG. 5, NNZ data of 4 × 4 blocks 10, 11, 14, and 15 of the seven prediction modes are transmitted, and the current macroblock is the macro of FIG. 5. In the case of block ③, NNZ data of blocks 4, 4, 5, 7, 13, and 15 of the seven prediction modes are transmitted.

According to the image codec system according to the second embodiment of the present invention described in detail above, an image encoding server performs an intra picture prediction encoding and an independent encoding on the entire image in parallel, thereby having two different values. Video bitstreams. The difference between the prediction mode data and the NNZ data used for the intra prediction encoding, and the decoded pixel values (differential pixel data) generated according to the two different encoding methods are separately stored as M-Data. When the user requests data for a target image in which a specific part of the entire image is specified, the image encoding server may include first image bits for macroblocks positioned on the leftmost, uppermost, and rightmost sides of the target image. The stream and the differential pixel data are transmitted, and the prediction mode data and the NNZ data are transmitted to the image decoding apparatus along with the second image bitstream for the remaining macroblocks of the target image. The apparatus for decoding an image decodes macroblocks located at the leftmost, uppermost, and rightmost sides of the target image by using the received first image bitstream and the differential pixel data, and performs a second image bitstream on the remaining macroblocks. , The prediction mode data, and the NNZ data are decoded to restore the entire target image.

According to the video codec system according to the second embodiment of the present invention, since only the first and second video bitstreams and M-Data are transmitted with respect to the target video, transmission efficiency can be improved. In addition, the image decoding apparatus does not need to perform decoding on an unnecessary portion of the image. As a result, since the video codec system according to the second embodiment of the present invention can efficiently use a limited transmission channel while supporting a spatial random access, it is recommended that a user view a high-definition video even for a target video that is a part of the entire video. It is possible.

As described above, according to the video codec system supporting the spatial random access and the video encoding / decoding apparatus and method constituting the same according to the present invention, a bitstream for the target video requested by the user and additional data necessary for decoding the same Even if the P-Data or the M-Data is minimized and transmitted to the decoding apparatus, the user can watch a high quality image with respect to a target image having an arbitrary position and size in the entire image. Therefore, according to the present invention, it is possible to efficiently use a limited transmission channel because it minimizes the amount of transmission data while supporting spatial random access. It is possible to watch high-definition images for the specific part desired.

Claims (19)

An image encoding apparatus supporting spatial random access to a target image that is a part of an entire image, An image encoding unit for generating an image bitstream by performing intra prediction encoding on the entire image; A prediction data memory for storing prediction data (P-Data) used to generate a prediction block for the current block of the entire image in an intra prediction encoding process; And And an extractor for extracting the image bitstream and the prediction data for the target image from the image bitstream for the entire image and the prediction data stored in the prediction data memory, respectively. The image encoding apparatus of claim 1, wherein the prediction data includes a decoded pixel value of a pixel adjacent to the current block. The data storage device of claim 2, wherein the prediction data memory stores prediction mode data of a block adjacent to the current block and / or number of non-zero coefficients (NNZ) data of a block adjacent to the current block. And a video encoding apparatus supporting a spatial random access. The image extractor of claim 1, wherein the extractor comprises: an image bitstream extractor for extracting an image bitstream of the target image from among image bitstreams of the entire image, and prediction data of the target image among prediction data of the entire image; And a predictive data extractor for extracting the data. The spatial random access method of claim 4, wherein the predictive data extractor extracts predicted data of the uppermost, leftmost, and / or rightmost macroblocks of the target image from the predictive data of the entire image. Image encoding apparatus that supports the. delete delete delete An image encoding apparatus supporting spatial random access to a target image that is a part of an entire image, A first image encoding unit for generating a first image bitstream by independently encoding the entire image in units of blocks having a predetermined size; A second image encoding unit for generating a second image bitstream by performing intra prediction encoding on the entire image; Inconsistent data that is a difference value between the decoded first pixel value of the first image bitstream generated by the first image coding unit and the decoded second pixel value of the second image bitstream provided by the second image coding unit. A mismatch data memory for obtaining and storing (Mismatch Data, M-Data) in units of blocks; For the uppermost, leftmost, and / or rightmost block of the target image, an image bitstream is extracted from the first image bitstream, and the uppermost, leftmost, and rightmost blocks among the blocks of the target image are extracted. A video bitstream extractor for extracting a bitstream from the second video bitstream for the non-blocks; And And an inconsistency data extractor for extracting the inconsistency data for the uppermost, leftmost, and / or rightmost blocks of the target image. 10. The image encoding apparatus of claim 9, wherein the difference value for the pixels located in the lowermost column and the rightmost row of each block is stored in the mismatched data memory. 10. The method of claim 9, wherein the mismatched data memory further includes prediction mode data of an adjacent block and / or data of number of non-zero coefficients (NNZ) of an adjacent block. Image encoding apparatus that supports the. An image decoding apparatus supporting spatial random access to a target image that is a part of an entire image, The first region of the target image performs independent decoding in units of blocks having a predetermined size, and the second region of the target image performs intra prediction prediction decoding in units of blocks of a predetermined size, and the intra region of the second region is in-screen. If the decoded pixel value of the neighboring block used for predictive decoding is a pixel belonging to the first region, a space including an image decoding unit for performing intra prediction prediction decoding after adding mismatched data to the independent decoded pixel value Image decoding apparatus that supports random access. The image decoding apparatus of claim 12, wherein the first area includes blocks positioned at the top, leftmost, and / or rightmost sides of the target image. The image of claim 13, wherein the pixel value to which the mismatch data is added is a pixel value of pixels located in the rightmost row and / or the lowest column in each block belonging to the first region. Decryption device. An image codec system comprising the image encoding device of claim 9 and the image decoding device of claim 12. An image encoding method for supporting a spatial random access to a target image that is a part of an entire image, Storing prediction data (Prediction Data, P-Data) used to generate a prediction block in the process of performing the intra prediction encoding by generating an image bitstream by performing intra prediction encoding on the entire image. Image encoding step; And And extracting and transmitting the image bitstream and the prediction data of the target image from the image bitstream and the prediction data, respectively, when the user requests a transmission of the target image. Coding method. An image decoding method for supporting a spatial random access to a target image that is a part of an entire image, Receiving prediction data together with an image bitstream in the target image; Generating a prediction block using the prediction data; And And reconstructing the target image using the image bitstream and the prediction block. An image encoding method for supporting a spatial random access to a target image that is a part of an entire image, Generating a first image bitstream by independently encoding the entire image in units of blocks having a predetermined size; Generating a second image bitstream by performing intra prediction encoding on the entire image; Obtaining and storing mismatch data (M-Data), which is a difference value between the decoded first pixel value of the first video bitstream and the decoded second pixel value of the second video bitstream, in block units; When there is a request for transmission of the target video from a user, among the first video bitstream, the second video bitstream, and the mismatched data, the uppermost, leftmost, and / or rightmost block of the target video is the same. Extracting an image bitstream from a first image bitstream, extracting an image bitstream from the second image bitstream for blocks other than the uppermost, leftmost, and rightmost blocks among the blocks of the target image; And extracting and transmitting the mismatched data with respect to the uppermost, leftmost, and / or rightmost blocks of the target image. An image decoding method for supporting a spatial random access to a target image that is a part of an entire image, Receiving a first video bitstream, a second video bitstream, and mismatch data in the target video; Restoring pixel values of the first region of the target image by decoding the first image bitstream; Correcting a pixel value for the restored first region by using the mismatched data; Restoring the pixel value of the second region of the target image by performing intra prediction prediction decoding using the corrected pixel value of the first region and the second image bitstream. Supported video decoding method.

Images