JP2007235314A - Coding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that tha coding efficiency is reduced when a region of interest is set to an image and the image is hierarchically coded. <P>SOLUTION: A resolution conversion section 12 reduces image data of a received frame in matching with a spatial resolution in each layer and gives the image data of each layer to a fundamental layer processing block 120 and an extended layer processing block 110. A ROI setting section 14 sets a ROI onto a moving picture frame in the unit of a layer. The fundamental layer processing block 120 divides the image of the fundamental layer whose resolution is converted into a low resolution by the resolution conversion section 12 according to ROI information of the fundamental layer and applies compression coding to the image by each ROI and outputs a result to a multiplexer section 18. The extended layer processing block 110 divides the image of the extended layer with a high resolution according to ROI information of the extended layer and applies compression coding to the image by each ROI and outputs the result to the multiplexer section 18. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding method for encoding an image, and more particularly to an encoding method for hierarchically encoding an image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. Also, in one stream, images with different image quality (for example, high and low image quality), different resolution (for example, high and low resolution), and different frame rates (for example, high and low frame rates) depending on the code amount H. can be compressed and decompressed. There is a next-generation image compression technique such as SVC (Scalable Video Coding), which is being standardized as an extension of H.264 / AVC.

  SVC, the next-generation image compression technology, encodes moving images with various scalability such as spatial scalability, temporal scalability, and SNR scalability so that moving images can be played at multiple different resolutions, frame rates, and image quality. Turn into. Coding can be performed by arbitrarily combining these scalability, and the scalability function of SVC is very flexible.

  One of SVC Requirements is Interactive ROI (Interactive Region of Interest; IROI) coding. ROI coding is a technique for coding a region of interest (ROI) of an image with a different image quality from other regions. In contrast, SVC interactive ROI encoding allows the user to specify the position and size of a region of interest on the screen sequentially while viewing a moving image, and reproduces the region of interest with different quality. It is what makes it possible. In SVC, since a moving image is encoded with various scalability, it is possible to decode a region of interest designated by the user at the time of reproduction with a quality different from that of other regions.

Patent Document 1 discloses a hierarchical image encoding technique that can maintain the quality of a reproduced image even in a communication environment in which packet loss and bandwidth fluctuation occur by encoding images hierarchically.
JP 2005-79953 A

  In SVC, an image is divided into a base layer and an enhancement layer and encoded hierarchically. When a region of interest is set in an image, the region of interest is set in common for all layers, and each layer is divided into regions according to the region of interest and independent coding is performed for each region. Efficiency decreases and processing load also increases.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an encoding technique capable of hierarchically encoding an area in an image.

  In order to solve the above-described problem, an encoding method according to an aspect of the present invention is configured such that, when an image is divided into a plurality of layers and encoded hierarchically, an area is set independently for each layer, and the area is set in each layer. Independent encoding is performed every time.

  Here, the “image” (picture) may be one independent still image or one of images arranged in a time series constituting a moving image. An "image" (picture) is a unit of encoding, and its concept includes a frame, a field, a VOP (Video Object Plane), and the like.

  According to this aspect, at the time of hierarchical encoding of an image, it is possible to generate encoded data obtained by encoding an image by setting a region independently for each layer, thereby reducing encoding efficiency and processing efficiency. Independent encoding can be performed in units of regions set in the image.

  In the case where any two of the plurality of regions set in the image have overlapping portions with each other (in this case, these plurality of regions are referred to as “having overlapping portions with each other”), The areas may be set separately for different layers. Thereby, overlapping of regions can be avoided in the same layer, and it is not necessary to separately handle and encode the overlapping portion, thereby preventing a decrease in encoding efficiency.

  For example, if the first region and the second region have overlapping portions, and the second region and the third region also have overlapping portions, but the first region and the third region do not overlap, The first area and the second area are set in different layers, and the second area and the third area are also set in different layers. Since the first area and the third area do not overlap, they may be set to the same layer. Therefore, in this case, it is sufficient that there are at least two layers. For example, by setting the second area in the first layer and setting the first and third areas in the second layer, There is no overlap in the area set at.

  As another example, when any two of the first to third regions overlap each other, that is, the first and second regions overlap each other, and the second and third regions When the regions also have overlapping portions, and the first and third regions also have overlapping portions, the first, second, and third regions are set to different layers. For example, the first area is set to the first layer, the second area is set to the second layer, and the third area is set to the third layer. Thereby, there is no overlap of regions in any layer.

  The plurality of layers may be a basic layer and an enhancement layer other than the basic layer in scalable hierarchical coding. The entire image may be encoded without setting a region in the base layer, and a region may be set in the enhancement layer, and independent encoding may be performed for each region.

  Scalable hierarchical coding is the coding of images hierarchically with scalability. For example, encoding with different playback quality of moving images such as spatial resolution, frame rate and image quality level, and multiple playback qualities. In other words, a moving image that is scalable hierarchically encoded in this manner includes a generation of encoded data of a level, and has scalability that an arbitrary reproduction quality level can be selected and decoded. Video encoded with different spatial resolution has spatial scalability, and video encoded with different frame rates has temporal scalability and is encoded with different image quality levels. A moving image has SNR scalability.

  When the encoded data of a plurality of reproduction quality levels is multiplexed with a hierarchical structure, for example, when only the encoded data of the lower layer is decoded, a moving image is reproduced at a low reproduction quality level and includes the encoded data of the upper layer. When decoded, the moving image is reproduced at a high reproduction quality level.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, a region where an image is set can be efficiently hierarchically encoded.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 100 according to the present embodiment conforms to SVC (Scalable Video Coding), which is a next-generation image compression technology, to spatial (spatial) scalability, temporal scalability, and SNR (signal to noise) for moving images. ratio) Perform “scalable coding” in which coding is performed with at least one of scalability.

  For the coding of moving images, the standards (MPEG-1, MPEG-2 and MPEG-2) of the MPEG (Moving Picture Experts Group) standardized by ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) MPEG-4), an H.264 standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) which is an international standard organization for telecommunications. 26x series standards (H.261, H.262 and H.263), or H.264, the latest video compression coding standard standardized jointly by both standards organizations. H.264 / AVC (the official recommendation names in both organizations are MPEG-4 Part 10: Advanced Video Coding and H.264, respectively) are used.

  In the embodiment, a frame is used as an example of a moving image encoding unit, but the encoding unit may be a field. The unit of encoding may be a VOP in MPEG-4.

  The encoding apparatus 100 receives an input of a moving image in units of frames, performs scalable encoding of the moving image, and outputs an encoded stream of the moving image. The input moving image frame is stored in the frame memory and read / written by each processing unit related to encoding.

  The encoding device 100 includes an enhancement layer processing block 110 and a base layer processing block 120 for encoding a moving image with spatial scalability, and the base layer processing block 120 compresses and encodes the moving image at a low resolution. The enhancement layer processing block 110 compresses and encodes the moving image with high resolution. Thereby, encoded data of moving images having different spatial resolutions is generated for each layer.

  Also, the encoding apparatus 100 uses an MCTF (Motion Compensated Temporal Filtering) technique in order to encode a moving image with temporal scalability. The MCTF technique combines subband division in the time axis direction with motion compensation, and performs hierarchical motion compensation. As a result, encoded data of moving images having different frame rates for each layer is generated.

  Also, the encoding apparatus 100 compresses and encodes a moving image by changing the quantization step and the number of lower bits to be discarded by the quantization in order to encode the moving image with SNR scalability. Thereby, encoded data of moving images having different image quality for each layer is generated.

  Note that spatial scalability, temporal scalability, and SNR scalability may be arbitrarily combined.

  The ROI setting unit 14 sets the ROI area in units of layers on the moving image frame. The ROI setting unit 14 can set an interactive ROI area in addition to a normal ROI area having no interactivity. In the interactive ROI area, the ROI area can be arbitrarily set when a moving image is reproduced. Hereinafter, when the interactive ROI area and the normal ROI area are collectively referred to, they are simply referred to as the ROI area.

  The attention area such as the interactive ROI area and the normal ROI area may be selected by the user specifying a specific area on the image, or a predetermined area such as the center area of the image may be selected. In addition, an important area such as an area in which a person or a character is shown may be automatically extracted as the attention area. Further, the attention area may be automatically selected in units of frames by tracking the movement of a specific object or the like in the moving image.

  It should be noted that the attention area is not necessarily intended only for reproduction with high image quality. For example, for the purpose of protecting privacy, it is necessary to reproduce a region of interest in which a person's face is captured with low image quality. Interactive ROI encoding and normal ROI encoding are also used for such purposes. By using scalable encoded image data, an area that needs privacy protection in the interactive ROI area can be reproduced at a low resolution, a low frame rate, or a low image quality. It is also possible to designate an area requiring privacy protection as a normal ROI area and encode in advance with a lower resolution, frame rate or image quality than other areas.

  In the present embodiment, the ROI setting unit 14 can specify the ROI region independently for each layer. For example, the ROI area is specified in the base layer, but the area corresponding to the ROI area of the base layer may not be specified as the ROI area in the enhancement layer. Conversely, in the enhancement layer, an area that does not designate the ROI area in the base layer may be designated as the ROI area. When there are a plurality of enhancement layers, the ROI region can be designated independently in each enhancement layer. Of course, not only the ROI region is set independently for each layer, but a common region may be set as the ROI region through all the layers of the base layer and the extension layer.

  The ROI setting unit 14 uses information for designating the ROI region in units of layers (hereinafter referred to as “ROI region information”), the image dividing unit 10a and the variable length coding unit 30a of the enhancement layer processing block 110, and base layer processing. This is given to the image dividing unit 10b and the variable length coding unit 30b of the block 120.

  The resolution conversion unit 12 reduces the image data of the input frame in accordance with the spatial resolution in each layer, and provides the image data of each layer to the base layer processing block 120 and the enhancement layer processing block 110. The resolution converter 12 provides the base layer processing block 120 with a low resolution image and the enhancement layer processing block 110 with a high resolution image.

  The base layer processing block 120 divides the base layer image converted to the low resolution by the resolution conversion unit 12 according to the ROI region information of the base layer, performs compression encoding for each ROI region, and outputs the result to the multiplexing unit 18. The encoding process in the base layer processing block 120 differs depending on whether each area to be encoded is an interactive ROI area, a normal ROI area, or a non-ROI area.

  In the base layer, the base layer processing block 120 encodes the normal ROI region with a different spatial resolution, frame rate or image quality level, or a combination thereof than the non-ROI region. For example, when encoding a normal ROI region with a higher image quality than a non-ROI region, for the normal ROI region, a different quantization table is used during quantization, and the applied quantization step is reduced. Encoding is performed with higher image quality than the non-ROI area by securing a larger number of effective bits, for example, by reducing the number of lower bits to be discarded by quantization.

  In general, the ROI region may have a plurality of different spatial resolutions, frame rates or image quality levels, or a combination thereof by scalable coding. One spatial resolution, frame rate or image quality level without scalable coding, Or you may just give these combinations.

  For the interactive ROI region, scalable coding is usually performed. As a result, only the area designated by the user in the interactive ROI area is reproduced with a higher resolution, frame rate, or image quality level, and other areas are reproduced with normal quality. Can do.

  When performing temporal scalable coding, the MCTF unit 20b operates in the base layer processing block 120, and coding is performed with different frame rates for each layer. When performing spatial scalability encoding, the enhancement layer processing block 110 operates in addition to the base layer processing block 120, and encoding with different spatial resolution is performed for each layer. When performing SNR scalable coding, coding with different image quality for each layer is performed by changing the quantization step and the number of lower bits to be discarded by quantization.

  For the non-ROI region, normally, scalable coding is not performed, the MCTF unit 20b related to temporal scalable coding does not operate in the base layer processing block 120, and spatial scalable coding using the enhancement layer processing block 110 There is no conversion.

  Each configuration of the base layer processing block 120 will be described.

  The image dividing unit 10b receives the frame image data of the base layer from the resolution conversion unit 12, and receives the ROI region information set for the base layer from the ROI setting unit 14. The image dividing unit 10b divides the input frame region into a plurality of small regions in accordance with the ROI region information of the base layer given from the ROI setting unit 14. A slice is used as an example of the small area. The slices are H.264. It is a basic unit of encoding in H.264 / AVC, and one frame can be divided into a plurality of slices and encoded in units of slices. In the present embodiment, as the ROI area is designated independently for each layer, the slice is also set independently for each layer.

  The image dividing unit 10b gives information (referred to as “slice information”) related to slice division of the base layer image to the variable length encoding unit 30b. The slice information includes information indicating the time of the slice group and slice area information.

  When only the interactive ROI area is set as the ROI area in the basic layer by the ROI setting unit 14, the entire area of the image of the basic layer includes the interactive ROI area and other areas (hereinafter referred to as “non-ROI area”). And divided. The non-ROI region becomes one slice, and the interactive ROI region is further divided into a plurality of slices in the interactive ROI region in order to have interactive properties.

  When both the interactive ROI area and the normal ROI area are set as the ROI areas in the base layer by the ROI setting unit 14, the entire area of the base layer image is the interactive ROI area, the normal ROI area, and the non-ROI area. One slice is set in the normal ROI area, another slice is set in the non-ROI area, and a plurality of slices are set in the interactive ROI area.

  The base layer processing block 120 encodes each slice independently without depending on other slices. That is, each slice is encoded using only the information closed in the encoding target slice without using the pixel data and motion vector information of the other slices.

  The reason why the interactive ROI region is independently encoded in units of slices is to enable decoding by designating a partial region as an ROI region in units of slices within the interactive ROI region. Assuming that the interactive ROI area is divided into 4 parts vertically and horizontally and includes 16 slices, independent encoding is performed in units of slices in the interactive ROI area. Arbitrary slices can be selected and reproduced with different qualities using the scalable encoded data for the selected slices.

  For example, when a high-quality image is requested for a designated area in the interactive ROI area, first, only the lowest layer is decoded for all slices in order to obtain an image with the lowest image quality. Next, only the slice corresponding to the area specified by the user is repeatedly decoded while going up the SNR scalability hierarchy until the image quality requested by the user is obtained.

  When an enlarged image is requested for a designated area in the interactive ROI area, only the lowest layer is decoded for all slices in order to obtain an image with the lowest image quality. Next, only the slice corresponding to the area designated by the user is repeatedly decoded while going up the spatial scalability hierarchy until the resolution requested by the user is reached.

  Since the basic layer processing block 120 does not have to be interactive in which the position and size of the region of interest can be arbitrarily specified unlike the interactive ROI region in the case of a normal ROI region and a non-ROI region, The entire ROI area and the entire non-ROI area are allotted to one slice without being divided into two. Of course, the normal ROI region and the non-ROI region may be divided into slices and encoded as necessary for the purpose other than the interactive property.

  The image dividing unit 10b supplies the frame layer image data to the MCTF unit 20b in units of slices. When the slice is time-scalable encoded, the MCTF unit 20b operates. The MCTF unit 20b performs motion compensation time filtering according to the MCTF technique. The MCTF unit 20b obtains a motion vector from the moving image frame, and performs temporal filtering using the motion vector. Temporal filtering is performed using a Haar wavelet transform, and as a result, the temporal filtering is decomposed into a plurality of layers having different frame rates including a high frequency frame and a low frequency frame in each layer. The decomposed high-frequency frame and low-frequency frame are stored in the memory for each layer, and the motion vector is also stored in the memory for each layer.

  When the processing in the MCTF unit 20b is completed, the high frequency frames of all layers and the low frequency frames of the final layer are sent to the prediction unit 24b, and the motion vectors of all layers are sent to the motion encoding unit 22b. .

  The prediction unit 24b performs intra-frame prediction of an image frame, and provides an intra-frame prediction error image to the DCT unit 26b. The DCT unit 26b performs discrete cosine transform (DCT) on the intra-frame prediction error image supplied from the prediction unit 24b, and gives the obtained DCT coefficient to the quantization unit 28b. The quantization unit 28b quantizes the DCT coefficient and provides it to the variable length coding unit 30b.

  The variable length encoding unit 30b receives the ROI region information of the base layer from the ROI setting unit 14, receives the slice information of the base layer from the image dividing unit 10b, and receives the quantized DCT coefficient of the difference image from the quantization unit 28b. receive. The variable length coding unit 30 b performs variable length coding on the ROI region information of the base layer, the slice information of the base layer, and the DCT coefficient, and provides the multiplexed unit 18 with the variable length coding. The slice information is necessary to specify a slice group and an area of each slice when decoding a frame image. The ROI area information is necessary for specifying a normal ROI area, an interactive ROI area, and a non-ROI area at the time of decoding.

  When performing SNR scalable encoding, encoded data with different image quality is generated for each layer by changing the number of lower-order bit planes to be discarded or changing the quantization step.

  The motion encoding unit 22 b encodes the motion vector information given from the MCTF unit 20 b and provides the same to the multiplexing unit 18.

  For spatial scalable coding, the motion coding unit 22b and the prediction unit 24b of the base layer processing block 120 respectively perform motion coding of the enhancement layer processing block 110 on the motion vector of each frame and the intra-frame prediction error image in the base layer. To the unit 22a and the interpolation processing unit 32.

  Next, each configuration of the enhancement layer processing block 110 will be described.

  The image dividing unit 10a receives the frame image data of the enhancement layer from the resolution conversion unit 12, and receives the ROI region information set for the enhancement layer from the ROI setting unit 14. The base layer image has a low resolution, whereas the enhancement layer image has a high resolution. The image dividing unit 10 a divides the input frame region into a plurality of slices according to the ROI region information of the enhancement layer given from the ROI setting unit 14. Since different ROI regions are set in the base layer and the enhancement layer, different slice divisions are performed in the enhancement layer and the base layer.

  The encoding process of each slice of the enhancement layer by the enhancement layer processing block 110 is basically the same as the encoding process of each slice of the base layer in the base layer processing block 120, and independent coding is performed for each slice. However, the enhancement layer processing block 110 encodes only the difference information between the base layer and the enhancement layer using the prediction coding result of the base layer processing block 120.

  Here, since the ROI area to be set is different between the base layer and the enhancement layer, when performing differential encoding between layers, it is noted that areas corresponding to each other are not the same ROI area. For example, even in the ROI region in the enhancement layer, the corresponding base layer region is a non-ROI region, and conversely, in the enhancement layer and the non-ROI region, the corresponding base layer region is an ROI region. Therefore, the enhancement layer processing block 110 takes a difference from the region of the base layer corresponding to the region of each slice of the enhancement layer at the time of differential encoding of each slice of the enhancement layer.

  The MCTF unit 20a of the enhancement layer processing block 110 performs the same motion compensation time filtering as the MCTF unit 20b of the base layer processing block 120 on each slice of the enhancement layer image, and encodes the motion vector information to the motion encoding unit 22a. Data is given to the prediction unit 24a. The motion encoding unit 22a of the enhancement layer processing block 110 receives information on the motion vector of the base layer image from the motion encoding unit 22b of the base layer processing block 120. The motion encoding unit 22a of the enhancement layer processing block 110 performs differential encoding between the motion vector information of each slice of the enhancement layer and the motion vector information of the corresponding region of the base layer, and is differentially encoded between layers. The obtained motion vector information is provided to the multiplexing unit 18.

  When motion vector information is differentially encoded between the base layer and the enhancement layer, the motion vector in the base layer is expanded to match the resolution of the enhancement layer. For example, when the height and width of the enhancement layer region are twice the height and width of the corresponding region of the base layer, the motion vector obtained for the corresponding region of the base layer is expressed in the height direction and the width direction. Double each. The motion encoding unit 22a of the enhancement layer processing block 110 encodes the difference between the motion vector of the base layer and the motion vector of the enhancement layer that have been expanded according to the resolution of the enhancement layer in this way. . By encoding the motion vector information between layers in this way, the amount of code of the motion vector information can be reduced rather than encoding the motion vector information of each area of the enhancement layer as it is.

  The interpolation processing unit 32 receives a prediction error image of each region of the base layer from the prediction unit 24b of the base layer processing block 120, and performs a process of interpolating pixels to match the resolution of the enhancement layer. The interpolation processing unit 32 gives the prediction error image of the base layer subjected to the interpolation processing to the prediction unit 24a of the enhancement layer processing block 110.

  The prediction unit 24a of the enhancement layer processing block 110 performs intraframe prediction encoding on the image frame provided from the MCTF unit 20a. Further, the prediction unit 24 a of the enhancement layer processing block 110 performs differential encoding between the prediction error image of the enhancement layer and the prediction error image of the base layer that is interpolated to match the resolution of the enhancement layer. By performing differential encoding of prediction error images between layers, the amount of codes can be reduced.

  The processing by the DCT unit 26a and the quantization unit 28a of the enhancement layer processing block 110 is the same as the processing by the DCT unit 26b and the quantization unit 28b of the base layer processing block 120.

  The variable length coding unit 30a of the enhancement layer processing block 110 receives enhancement layer ROI region information from the ROI setting unit 14, receives enhancement layer slice information from the image segmentation unit 10a, and receives a prediction error image from the quantization unit 28a. Receive quantized DCT coefficients. The variable length coding unit 30 a performs variable length coding on the enhancement layer ROI region information, enhancement layer slice information, and DCT coefficients, and supplies the result to the multiplexing unit 18.

  The multiplexing unit 18 generates an encoded stream in which the encoded data in the base layer given from the base layer processing block 120 and the coded data in the enhancement layer given from the enhancement layer processing block 110 are combined into one. Output. The encoded data of each layer includes image data, motion vector information, ROI region information, and slice information.

  In this embodiment, the ROI region information and slice information of each layer are encoded by the variable length encoding units 30a and 30b. However, the ROI region information and slice information of each layer are multiplexed without being encoded. It may be given to the unit 18 and added to the header of the encoded stream.

  In the above description, the base layer processing block 120 and the enhancement layer processing block 110 are separately provided, and the configuration for encoding the base layer low resolution image and the enhancement layer high resolution image has been described. Components common to the enhancement layer processing block 110 may be shared between the base layer and the enhancement layer. For example, only the configuration of the base layer processing block 120 is provided, the base layer is encoded in the base layer processing block 120, and the prediction error image and motion vector information in the base layer are held in the memory. Next, using the base layer encoding result stored in the memory, the enhancement layer encoding process is executed in the base layer processing block 120. Thus, if the configuration of the encoding process in the base layer is diverted to the enhancement layer, the circuit scale of the encoding device 100 can be reduced.

  In the above description, the case where there are two layers of the spatial scalability, that is, the base layer and the enhancement layer has been described, but three or more layers of spatial scalability may be provided. In that case, the base layer processing block 120 is provided for the lowest layer, and the configuration of the extended layer processing block 110 is provided for each of the other layers. The prediction error image and motion vector information are sent from the lower layer to the upper layer, and differential encoding is performed in each layer. Alternatively, only the base layer processing block 120 may be provided, and the base layer processing block 120 may be repeatedly used for each layer so that each layer is sequentially encoded.

  Hereinafter, an example in which an image is encoded by setting an ROI region in units of layers by the encoding device 100 will be described.

  First, for comparison, a case will be described in which an image is encoded without setting an ROI region for each layer. FIG. 2 shows an example in which ROI regions common to both the base layer and the enhancement layer are set and encoded. A normal ROI area 202a and an interactive ROI area 204a are set in the base layer 200a of the image, and a normal ROI area 202b and an interactive ROI area 204b are also set in the same position in the extended layer 200b.

  As described above, when an ROI area is designated for an image in SVC, an ROI area common to both the base layer and the enhancement layer is normally set, and the image is divided into slices in accordance with the ROI area in each layer. The

  In the example of FIG. 2, in each layer, an image is divided into normal ROI areas 202a and 202b, interactive ROI areas 204a and 204b, and other non-ROI areas, and a slice is assigned to each area. In addition, the overlapping area (the hatched area) of the normal ROI areas 202a and 202b and the interactive ROI areas 204a and 204b is processed as an independent area different from the normal ROI areas 202a and 202b and the interactive ROI areas 204a and 204b. . In this way, the image is divided into four areas, that is, a normal ROI area, an interactive ROI area, a non-ROI area, and an overlapping area of the normal ROI area and the interactive area in each layer, and slices are assigned. Note that the interactive ROI areas 204a and 204b are further divided into small areas in order to provide interactivity, and a slice is assigned to each small area.

  As described above, in the normal SVC, when the ROI region is set, the same region division is performed for all layers, and the coding is performed in units of regions according to the region division for each layer. The load also increases. Therefore, in the present embodiment, the setting of the ROI area is made different for each layer.

  FIGS. 3A and 3B show an example in which the ROI region is set differently in the base layer and the enhancement layer. In FIG. 3A, the ROI area is not set in the base layer 200a, and the normal ROI area 202b is set only in the enhancement layer 200b. Since the ROI region is not set in the base layer 200a, the entire region of the image can be encoded without dividing the region, and the efficiency of differential encoding within the frame can be increased.

  On the other hand, since the normal ROI area 202b is set in the enhancement layer 200b, the normal ROI area 202b and the other non-ROI areas are divided into two and encoded independently for each area. Also, differential encoding between layers is performed by taking the difference between the normal ROI region 202b and the region at the same position of the base layer 200a.

  In FIG. 3B, the normal ROI area 202a is set in the base layer 200a, and the interactive ROI area 204b is set in the enhancement layer 200b. The base layer 200a is divided and encoded into a normal ROI area 202a and other non-ROI areas, and the enhancement layer 200b is divided into an interactive ROI area 204b and other non-ROI areas. Encoded.

  FIG. 2 is compared with FIG. In FIG. 2, the base layer 200a and the enhancement layer 200b must be divided into four regions including the overlapping portion of the ROI region and the interactive ROI region. On the other hand, as shown in FIG. 3B, if the ROI area is set in the base layer 200a and the interactive ROI area is set in the enhancement layer 200b, it is necessary to distinguish and encode the overlapping part of the ROI area and the interactive ROI area. Both the base layer 200a and the enhancement layer 200b need only be divided into two regions and encoded. For this reason, encoding efficiency can be improved and processing load can be reduced.

  In the example of FIG. 3A, since the normal ROI region 202b is provided in the enhancement layer 200b, the normal ROI region has spatial scalability, and the region can be displayed with high resolution.

  In the example of FIG. 3B, since the interactive ROI area 204b is provided in the enhancement layer 200b, the interactive ROI area has spatial scalability and can be displayed with high resolution, but the normal ROI area 202a is in the base layer 200a. Since it is set, the normal ROI area has no spatial scalability and can only display at low resolution. Therefore, in the example of FIG. 3B, the number of layers may be further increased so that the normal ROI region also has spatial scalability.

  FIG. 4 shows an example of encoding with different ROI region settings in three layers. Compared to the example of FIG. 3B, the number of layers is increased by one, the ROI area is not set in the base layer 200a, the normal ROI area 202b is set in the first extension layer 200b, and the second extension layer 200c is set. An interactive ROI area 204c is set. The base layer 200a is a low resolution image, the first enhancement layer 200b is a medium resolution image, and the second enhancement layer 200c is a high resolution image. Thus, by setting the ROI area in three layers, it is possible to provide spatial scalability for both the normal ROI area and the interactive ROI area.

  5 to 8 show other examples of setting ROI areas in units of layers. 5 to 8, for the sake of simplicity, the normal ROI region and the interactive ROI region are not distinguished from each other and are simply described as ROI regions. In either case, only the normal ROI region is set, only the interactive ROI region is set, Any setting in which a normal ROI area and an interactive ROI area are mixed may be used.

  FIG. 5 shows an example in which the ROI area is not set in the base layer but the ROI area is set in the enhancement layer. The ROI area is not set in the base layer 200a, the ROI area 210b is set in the first enhancement layer 200b, and the ROI area is also set in the second enhancement layer 200c at the same position as the ROI area 210b of the first enhancement layer 200b. 210c is set.

  When encoding the ROI area 210b of the first enhancement layer 200b, the difference from the corresponding area of the base layer 200a is encoded. When the ROI region 210c of the second enhancement layer 200c is encoded, the difference from the ROI region 210b of the first enhancement layer 200b is encoded because the ROI region 210b is located at the same position of the first enhancement layer 200b. To do. The area designated as the ROI area has scalability at three levels of spatial resolution. However, since the ROI area is not set in the base layer 200a, the base layer 200a can be encoded without being divided.

  FIG. 6 shows an example in which the ROI area set in units of layers has a nested structure between layers. The ROI area 220a is set in the base layer 200a, and the ROI area 222b wider than the ROI area 220a of the base layer 200a is set in the first enhancement layer 200b. In the second enhancement layer 200c, a ROI region 224c that is wider than the ROI region 222b of the first enhancement layer 200b is set.

  When the ROI region 222b of the first enhancement layer 200b is encoded, a difference between the ROI region 220a of the base layer 200a is calculated for the portion overlapping the ROI region 220a of the base layer 200a. For the outer portion that does not overlap the ROI region 220a, the difference from the non-ROI region of the base layer 200a is calculated. The same applies when the ROI region 224c of the second enhancement layer 200c is encoded.

  In FIG. 6, a wide ROI area including the ROI area specified in the base layer is specified in the extension layer so that the ROI area is nested. On the contrary, the narrow ROI area included in the ROI area specified in the base layer May be nested in the ROI area by designating in the enhancement layer. In the latter case, the center portion can be provided with scalability by a high-resolution image.

  FIG. 7 shows a setting example in which the position of the ROI region is different for each layer. The ROI area 230a is set in the base layer 200a, and the ROI area 232b is set in the first enhancement layer 200b at a position partially overlapping with the ROI area 230a of the base layer 200a. Further, the ROI region 234c is set in the second enhancement layer 200c at a position different from the ROI region 232b of the first enhancement layer 200b.

  Even when a plurality of ROI areas are designated at different positions, the ROI areas are set separately for each layer, so that the number of area divisions in each layer can be reduced. Also, by providing ROI regions for each layer, even if there are overlaps between layers in the ROI region, it is possible to reduce the number of ROI regions that overlap in the same layer. Efficiency can be increased.

  FIG. 8 shows an example in which different numbers of ROI regions are set for each layer. The first ROI regions 240a, 240b, 240c are provided in all layers of the base layer 200a, the first enhancement layer 200b, and the second enhancement layer 200c, and the second ROI regions 242b, 242c The third ROI area 244c is provided only in the second enhancement layer 200c, and is provided in the enhancement layer 200b and the second enhancement layer 200c.

  Although there are three ROI regions in the image of FIG. 8, only one ROI region is set and encoded in the base layer 200a, and only two ROI regions are set and encoded in the first enhancement layer 200b. Compared with the case where three ROI regions are set in all layers, the encoding efficiency is improved. In the example of FIG. 8, the case where the number of ROI areas increases as the hierarchy increases, that is, the resolution increases, but conversely, the lower the resolution, the larger the number of ROI areas may be set.

  As described above, according to coding apparatus 100 of the present embodiment, since an independent ROI region is set for each layer and assigned to a slice, the number of divided regions is reduced when viewed in each layer. Encoding efficiency can be improved and processing load can be reduced.

  Even when the ROI regions overlap each other on the image, if the overlapping ROI regions are divided and set in different layers, it is not necessary to assign the overlapping regions to different slices, so that the encoding efficiency does not decrease.

  As the ROI area, not only the normal ROI area but also the interactive ROI area can be set and encoded for each layer in the same manner, and the normal ROI area and the interactive ROI area are also encoded in the same way. be able to.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above embodiment, an example in which an image is divided into slices according to the ROI area set for each layer has been described. However, the present invention is not limited to ROI, and is widely used when an area is set for an image for some purpose. Can be applied. In addition, even when an area is not set in an image, the present invention is also effective when it is necessary to set and encode different slices for each layer. In SVC, it is normal to set and encode the same slice through all layers, and to give bits specifying the type of slice group only to the base layer, but encode by changing the shape and number of slices for each layer. If necessary, a bit specifying the slice group type may be given not only to the base layer but also to the enhancement layer.

  In the above embodiment, hierarchical encoding has been described by taking a moving image as an example, but the present invention can also be applied to hierarchical encoding of still images.

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure which shows the example which sets and encodes the ROI area | region common to both a base layer and an extended layer. It is a figure which shows the example encoded by changing the setting of a ROI area | region in a base layer and an extended layer. It is a figure which shows the example encoded by changing the setting of a ROI area | region in three layers. It is a figure which shows the example which sets an ROI area | region in an extended layer, without setting an ROI area | region in a base layer. It is a figure which shows the example in which the ROI area | region set per layer has a nesting structure between layers. It is a figure which shows the example of a setting from which the position of a ROI area | region differs for every layer. It is a figure which shows the other example of the setting of the ROI area | region of a layer unit.

Explanation of symbols

  10a, 10b Image segmentation unit, 12 Resolution conversion unit, 14 ROI setting unit, 18 Multiplexing unit, 20a, 20b MCTF unit, 22a, 22b Motion coding unit, 24a, 24b Prediction unit, 26a, 26b DCT unit, 28a, 28b quantization unit, 30a, 30b variable length coding unit, 32 interpolation processing unit, 100 coding device, 110 enhancement layer processing block, 120 base layer processing block.

Claims (6)

  An encoding method, wherein when an image is divided into a plurality of layers and encoded hierarchically, regions are set independently for each layer, and independent encoding is performed for each region in each layer.   The encoding method according to claim 1, wherein information specifying the region of each layer is included in encoded data of the image.   The encoding method according to claim 1 or 2, wherein a slice, which is an independent encoding unit in an image, is independently set for each layer according to the region set in each layer.   4. The method according to claim 1, wherein when any two of the plurality of regions set in the image have overlapping portions, the plurality of regions are set separately in different layers. An encoding method according to claim 1.   The encoding method according to claim 1, wherein the area is encoded with an image quality different from that of other areas.   The encoding method according to any one of claims 1 to 5, wherein the plurality of layers are a base layer and an enhancement layer other than the base layer in scalable hierarchical encoding.

Images