JP4321483B2 - Spatial resolution conversion method and spatial resolution conversion program - Google Patents

Description

  The present invention relates to a spatial resolution conversion method and a spatial resolution conversion program, and in particular, a space of an image frame constituting one of a video having a high-definition spatial resolution and a video having a lower spatial resolution. The present invention relates to a spatial resolution conversion method and a spatial resolution conversion program for converting resolution to the other spatial resolution.

  In the conventional scalable coding, as shown in FIGS. 18A to 18C, a spatial resolution is set such that a certain resolution is reduced by half both vertically and horizontally, and these spatial resolutions can be freely switched. In general, it is possible to achieve scalability in the spatial direction by configuring it as possible.

  Further, the spatial resolution that cannot be supported by the spatial resolution obtained by reducing the vertical and horizontal sides by half, for example, the spatial resolution from the HD resolution of 1920 pixels horizontally and 1080 pixels vertically to 720 pixels horizontally and 480 pixels vertically For conversion, cropping processing as shown by images 102 and 105 in FIG. 19, letterbox processing as shown by reduced images 108, 109 and 110 in FIG. 19, and squeeze as shown by images 502 and 503 in FIG. By performing the processing, scalable encoding and decoding are realized by performing spatial resolution conversion from HD (High Definition) resolution to SD (Standard Definition) resolution (for example, Patent Document 1, Patent Document 2, Patent) Reference 3).

  Many display devices having an aspect ratio of 16: 9 or 4: 3 realize various displays using more flexible spatial resolution conversion (see, for example, Patent Document 4).

Japanese Patent No. 33748989 JP-A-6-284413 Japanese Patent No. 3564087 JP-A-6-6634

  Conventionally, when scalable encoding and decoding are performed on an image signal, a predetermined spatial filtering process is performed on the high-resolution image signal and then down-sampling at a ratio of 2: 1 to obtain a low-resolution image. Generate a signal. For example, the HD resolution image signal is subjected to a predetermined spatial filtering process and then down-sampled at a ratio of 2: 1 to generate a low resolution image signal.

  When this low resolution image signal is encoded and transmitted, the aspect ratio of the image remains 16: 9 that of the original HD resolution image signal. Therefore, in an NTSC television receiver having an aspect ratio of 4: 3, there is a problem that it is impossible to monitor a 1/4 resolution image signal obtained by down-sampling from an HD resolution image signal. There are many studies on the problem, and the improvement is progressing.

  However, there are some problems in the spatial resolution scalable encoding using the spatial resolution conversion from the conventional HD resolution to the SD resolution.

  FIGS. 19A to 19C are examples in which cropping processing is used when performing spatial resolution conversion from HD resolution to SD resolution. In order to simplify the description here, unless otherwise specified, the resolution having 1920 pixels in the horizontal direction and 1080 pixels in the vertical direction (1920 × 1080 pixels) is hereinafter referred to as HD resolution. The resolution having 720 pixels in the horizontal direction and 480 pixels in the vertical direction (720 × 480 pixels) is the SD resolution.

  19, first, a more important area is identified from the image frame 101 of the HD resolution moving image shown in FIG. 19A, and the cropping process is performed on the area indicated by 102 in FIG. The area 102 is obtained by multiplying the SD resolution indicated by the area 105 shown in FIG. 5C by 2.25 in both the vertical and horizontal directions so that the previous pixel becomes the same pixel area as the vertical pixel of the HD resolution. This area is cropped from the HD resolution using the maximum pixel area that can be accommodated and can be expressed in the aspect ratio of the SD resolution.

  Then, in order to reduce the image frame indicated by 102 in FIG. 19B to the original SD resolution of 720 × 480 pixels, a predetermined filtering process for reducing the image frame by 1 / 2.25 times both vertically and horizontally is performed. By doing so, the SD resolution shown in FIG. The SD resolution moving image frame 105 obtained in this way can use the texture information in the region 102 cropped from the HD resolution moving image frame to the maximum, instead of 103, FIG. An area that cannot be expressed within the SD resolution as indicated by 104 occurs.

  FIG. 20 shows encoding processing between spatial resolutions in moving image encoding having spatial scalability using the cropping processing as shown in FIG. In general, in spatial scalable coding with spatial resolution conversion, encoding is performed by setting a base layer having a basic spatial resolution and an enhancement layer having additional information for improving the spatial resolution for the base layer. Do.

  This enhancement layer is composed of at least one layer. By using multiple enhancement layers, it is possible to achieve finer resolution scalability. However, overhead information for configuring multiple enhancement layers is required, which affects coding efficiency. is necessary. In FIG. 20, 201 in FIG. 20A is a base layer, and 206, 207, and 208 in FIG.

  As shown in FIG. 20, since the base layer 201 is encoded based on the SD resolution generated by the cropping process from the original HD resolution, the local decoded image obtained by decoding the base layer 201 after decoding is encoded. When the image is vertically and horizontally multiplied by 2.25, an image 202 shown in FIG. The image 202 is subjected to predictive coding between layers (in FIG. 20, the layer is expressed as a hierarchy) in spatial scalable coding.

  However, in the locally decoded image of the HD resolution layer shown in FIG. 20C, only the region 203 has a high correlation with the image 202, so the regions 204 and 205 can use the correlation between layers. Instead, it becomes a target region for intra coding, which is intra-frame coding. By performing the predictive coding between layers, as shown in FIG. 20D, the predictive code information 206 between layers resulting from the predictive coding of the region 202 and the region 203 and the region 204 are intra-coded. Resulting intra code information 207 and intra code information 208 resulting from intra coding of the region 205 are obtained. In predictive coding, inter-layer predictive coding additional information, which is various additional information including motion vector information and information indicating a state at the time of encoding, is obtained. These 206, 207, 208 and inter-layer predictive coding additional information become information for configuring the enhancement layer.

  As described above, when cropping processing is used for resolution conversion in spatial scalable coding, intra coding such as the intra code information 207 and 208 is accompanied, so that the coding efficiency is lowered and the coding and decoding are performed. It becomes a factor which increases the amount of calculation at the time.

  Further, since separate encoding such as inter-layer prediction encoding for obtaining the inter-layer prediction code information 206 shown in FIG. 20D and intra encoding for obtaining the intra-code information 207 and 208 is performed, the image quality of the decoding result is determined. Variation occurs. As a result, not only the subjective image quality of the decoding result is affected, but also the fact that the image quality is not uniform causes a reduction in coding efficiency when attempting to construct a higher enhancement layer.

  FIG. 21 shows encoding processing between spatial resolutions in moving image encoding having spatial scalability using the cropping processing as shown in FIG. 19 as in FIG. FIG. 21 is the same as FIG. 20 regarding the base layer. In FIG. 21, with respect to the enhancement layer, first, the first enhancement layer is constituted by the intra-coding result for the region 301 outside the cropping region shown in FIG. 21A, and decoding is performed again from the first enhancement layer. The local decoded images 302 and 303 obtained in the above are used.

  Further, in order to configure the second enhancement layer, predetermined enlargement processing is performed on the region 301 and the images 302 and 303 to obtain images 304, 305, and 306 as shown in FIG. Thereafter, inter-layer predictive coding is performed between the images 304, 305, and 306 and the local decoded image 307 illustrated in FIG. 21C, and the obtained result is used as the second enhancement layer, so that spatial scalability is achieved. Can be encoded.

  However, even with such a configuration, it is difficult to fundamentally improve the factors that can occur in the configuration of FIG.

  The image frame 101 in FIG. 19 (A), the reduced image 108 in FIG. 19 (D), the image frame 101, the image 109 in FIG. 19 (E), and the image 110 in FIG. This is an example in which letterbox processing is used when converting spatial resolution to resolution. The reduced image 108 is directly fixed from the image frame 101 with the aspect ratio fixed, and the HD resolution image frame 101 is subjected to predetermined spatial filtering processing and reduction processing to be 3/8 times both in the vertical and horizontal directions, and is stored in the SD resolution. Further, the upper and lower areas are supplemented with a predetermined pixel value, usually a pixel value for expressing black.

  An image 109 shown in FIG. 19 (E) has a predetermined size with respect to the image frame 101 in order to adjust the horizontal size of the HD resolution image frame 101 to a horizontal size of 2.25 times the SD resolution with a fixed aspect ratio. The vertical and horizontal areas are multiplied by 27/32 by a spatial filtering process and a predetermined reduction process, and the upper and lower areas are filled with a predetermined pixel value, usually a pixel value for expressing black. Thereafter, a predetermined spatial filtering process and a predetermined reduction process are performed on the image 109 shown in FIG. 19E, and the image is reduced by 1 / 2.25 both vertically and horizontally, thereby reducing the reduced image 110 shown in FIG. Is generated.

  The SD resolution moving image frames indicated by the reduced images 108 and 110 are generated by predetermined spatial filtering and reduction processing so as to be inscribed in the SD resolution by letterbox processing from the HD resolution moving image frames. Instead of being able to use the characteristics of the entire image frame 101, the texture information in the areas included in the reduced images 108 and 110, that is, in order to perform further reduction processing than the reduced image 105 shown in FIG. The high frequency component information for expressing the details of the image is less than that of the reduced image 105.

  FIG. 22 shows encoding processing between spatial resolutions in moving image encoding having spatial scalability using the letterbox processing as shown in FIG. In FIG. 22, 401 is a base layer and 405 is an enhancement layer. First, by removing pixels filled up and down from the SD resolution image 401 shown in FIG. 22 (A) and multiplying it by 8/3 both vertically and horizontally by a predetermined spatial filtering process, An image 403 shown in FIG. 22C is directly obtained from the image 401.

  Alternatively, the image 402 shown in FIG. 22B is obtained by multiplying the image 401 by 2.25 both vertically and horizontally by an enlargement process using a predetermined spatial filtering process. Thereafter, the pixels filled in the upper and lower portions are removed from the image 402, and an image 403 shown in FIG. 22C is obtained by multiplying both vertically and horizontally by 32/27 times by an enlargement process by a predetermined spatial filtering process.

  Next, in order to construct an enhancement layer of the image 405 shown in FIG. 22E, inter-layer predictive coding is performed between the image 403 and the HD resolution local decoded image 404 shown in FIG. By using the obtained inter-layer predictive coding information 405 shown in FIG. 5E as an enhancement layer, it is possible to perform moving image coding with spatial scalability.

  In such spatial scalable coding, when letterbox processing is used for resolution conversion, as shown in FIG. 22A, compared with the case where cropping processing is used for resolution conversion as shown in FIGS. Since there is little texture information included in the image 401, that is, high-frequency component information for expressing the details of the image, an image 403 shown in FIG. 22C obtained by enlarging the image 401 is temporarily Even when encoding and decoding are performed by applying lossless encoding so as not to affect the base layer encoding, the images 202, 204, 205 shown in FIG. 20 and the images shown in FIG. Compared with the images 304, 305, and 306, it becomes a blurred image with less fineness.

  Therefore, the use of letterbox processing for resolution conversion is a factor that degrades the image quality of the base layer decoded image obtained at the time of decoding as compared with the image quality of the base layer decoded image of FIG. 20 or FIG. It becomes. Further, when trying to maintain the same image quality as that of FIG. 20 or FIG. 21 using the enhancement layer, it is necessary to allocate a larger amount of code to the enhancement layer.

  23 (A), (B), (D) and FIGS. 23 (A), (C), (D) show examples of using squeeze processing when performing spatial resolution conversion from HD resolution to SD resolution. . The image 502 shown in FIG. 23C is subjected to a squeeze process by multiplying the image frame 501 of the HD resolution shown in FIG. 23A by 27/32 by a predetermined spatial filtering process and a predetermined reduction process in the horizontal direction. Is generated. Thereafter, by performing a predetermined spatial filtering process and a predetermined reduction process on the image 502, the reduced image 503 shown in FIG.

  Further, an image 504 shown in FIG. 23B is obtained by performing predetermined spatial filtering processing and predetermined reduction processing on the HD resolution image frame 501 shown in FIG. Generated by multiplying by 9. Thereafter, a reduced image 503 shown in FIG. 23D is generated by performing a squeeze process 27/32 times in the horizontal direction of the image 504 by a predetermined spatial filtering process and a predetermined reduction process.

  The SD resolution moving image frame indicated by the reduced image 503 thus obtained is generated by performing predetermined spatial filtering and reduction processing in the horizontal direction from the image frame 501 of the HD resolution moving image by squeeze processing. Therefore, instead of being able to use the characteristics of the entire image frame 501, a further reduction process is performed in the horizontal direction as compared with the reduced image 105 shown in FIG. The horizontal texture information in the region included in the reduced image 503 shown, that is, the high frequency component information for expressing the details of the image is less than that of the reduced image 105.

  Further, the reduced image 105 is an image obtained as a result of performing cropping processing on a more important area, and the aspect ratio of the image is maintained for the cropped area. In the case of the reduced image 503 generated by performing the squeeze process, as shown in FIG. 23D, since the horizontal reduction process is performed over the entire image, the aspect ratio of the original image is maintained. Cannot be obtained, and the entire image becomes distorted.

  FIG. 24 shows coding processing between spatial resolutions in moving image coding having spatial scalability using the squeeze processing as shown in FIG. In FIG. 24, 601 shown in (A) is a base layer, and 606 shown in (E) is an enhancement layer. First, an enlargement process by a predetermined spatial filtering process is performed on the SD resolution image 601 shown in FIG. 24A, and the image 603 shown in FIG. .

  Thereafter, in order to cancel the squeeze, the image 603 is subjected to enlargement processing by a predetermined spatial filtering process in the horizontal direction and multiplied by 32/27 to generate an image 604 shown in FIG. Alternatively, the image 602 shown in FIG. 24C is generated by canceling the squeezing of the SD resolution image 601 first, and then enlarged by a predetermined spatial filtering process to be 2.25 times both vertically and horizontally. As a result, an image 604 shown in FIG.

  Next, in order to construct an enhancement layer of the image 606, inter-layer predictive coding is performed between the image 604 and the local resolution image 605 having the HD resolution shown in FIG. The inter-layer predictive coding information 606 shown in FIG. 5) is image information of the enhancement layer, so that moving picture coding having spatial scalability can be performed.

  When squeeze processing is used for resolution conversion in such spatial scalable encoding, the horizontal direction included in the base layer image 601 is compared to the case where cropping processing is used for resolution conversion as shown in FIGS. 24, so that the image 604 obtained by the squeeze release and enlargement processing in FIG. 24 is not affected by the base layer encoding. Even when encoding and decoding are performed by applying lossless encoding, the horizontal direction is more subtle than 202, 204, 205 in FIG. 20 and 304, 305, 306 in FIG. The image will be blurred.

  Therefore, the use of squeeze processing for resolution conversion is a factor that degrades the image quality of the decoded image of the base layer obtained at the time of decoding as compared with the image quality of the decoded image of the base layer shown in FIGS. Become. Further, when trying to maintain the same image quality as that of FIG. 20 or FIG. 21 using the enhancement layer, it is necessary to allocate a larger amount of code to the enhancement layer.

  Furthermore, when trying to display the decoded image of the base layer during scalable decoding, the entire image is distorted because of the squeeze process, so when attempting to display correctly, the entire image is displayed. When the squeeze is canceled again and the display target has an SD resolution aspect ratio, a problem arises in that a cropping process must be performed on the display area.

  The present invention has been made in view of the above points, and provides a spatial resolution conversion method and a spatial resolution conversion program for enabling more efficient encoding and decoding when performing scalable encoding and decoding. For the purpose.

In order to achieve the above object, a spatial resolution conversion method according to a first aspect of the present invention provides a video signal having a first spatial resolution, in units of image frames constituting the video signal, more than the first spatial resolution. A spatial resolution conversion method for converting into a video signal having a low second spatial resolution,
Cropping, which is an image area for making a second image frame that is an image frame of a video signal having a second spatial resolution within a first image frame that is an image frame of a video signal having a first spatial resolution A first step of specifying an area, a first original area that is an area for performing spatial resolution conversion without distorting the image shape in the cropping area, and a first original area in the first image frame The second step of generating the first area information that is information for specifying the area and the area outside the first original area in the cropping area are areas for performing spatial resolution conversion by distorting the image shape. According to the third step of identifying the first distortion region and the ratio between the size of the second image frame and the size of the first image frame, the first orientation is determined. With the second original area by reducing the null region without distorting the image shape in the first reduction mode, the first original region outside the region in the first image frame, the first strained region The image shape is distorted by the second reduction method so as to become a region that matches the region reduced by the first reduction method, and the second distortion region is obtained by reducing the image shape. 2nd area information which is information for specifying 2 original areas is generated, and a 2nd image frame which has the 2nd spatial resolution is generated by the 2nd original area and the 2nd distortion area And a fourth step.

  In the present invention, the image information of the first area is directly converted into the second resolution without distorting the image information of the original area, and the image information of the first resolution of the area outside the original area in the clipping area is Since the image data is converted to the second resolution so as to be within the distortion area, all the image information in the image area of the first resolution can be converted to the second resolution, and the area important as the image information is the original. As a region, resolution conversion can be performed without distortion even when converting to the second resolution.

In order to achieve the above object, the second invention converts the first image frame of the video signal having the first spatial resolution by the spatial resolution conversion method of the first invention. The video signal of the second image frame having the second spatial resolution lower than the first spatial resolution is again converted into the video signal having the first spatial resolution in units of image frames constituting the video signal. A spatial resolution conversion method for converting to
The second image frame to be subjected to spatial resolution conversion is a first original area that is subjected to spatial resolution conversion in a first cropping area that is an image area for making a second image frame in the first image frame. A second original area obtained by reducing the image shape without distorting the image by the first reduction method according to a ratio between the size of the second image frame and the size of the first image frame; The image shape is determined by the second reduction method so that the area outside the first original area in the first image frame coincides with the area obtained by reducing the first distortion area by the first reduction method. And a second strain region obtained by distorting and shrinking,
A first step of specifying a second original area in the second image frame based on second area information which is information for specifying a second original area in the second image frame; A second step of identifying the image frame as a second cropping region, a third step of identifying a region outside the second original region within the second cropping region as a second distortion region, According to the ratio of the size of the second image frame and the size of the first image frame, the first original region is enlarged without distorting the image shape by the first enlargement method using the first enlargement method. together to restore, and the size of the first and second image frames, the first area information and second area information is information for specifying the first original region in the first image frame The basis of the second strained region, such that the first original area outside the realm Match region in the first image frame, to enlarge distort the image shape by the second expansion method A region outside the first original region is restored, and a first image frame having a first spatial resolution is restored by using the restored first original region and the restored region outside the first original region . These steps are provided.

  In this invention, the image frame converted from the first resolution to the second resolution in the first invention can be converted to the original image frame of the first resolution.

  In order to achieve the above object, a resolution conversion program according to a third aspect of the present invention provides a video signal having a first spatial resolution, in units of image frames constituting the video signal, from the first spatial resolution. A spatial resolution conversion program for causing a computer to execute a spatial resolution conversion for converting into a video signal having a lower second spatial resolution, and causing the computer to sequentially execute each step of the first invention. It is characterized by.

In order to achieve the above object, a resolution conversion program according to a fourth aspect of the present invention provides a first spatial resolution converted from a video signal having the first spatial resolution by the spatial resolution conversion method according to the first aspect. The computer executes spatial resolution conversion for converting a video signal having a lower second spatial resolution into a video signal having the first spatial resolution again in units of image frames constituting the video signal. A spatial resolution conversion program for causing a computer to sequentially execute the steps of the second invention.
Furthermore, in order to achieve the above object, the resolution converter of the fifth invention is characterized in that each step of the first invention is realized by hardware, and the resolution converter of the sixth invention is the first Each step of the invention is realized by hardware.

  According to the present invention, all the image information in the image area of the first resolution is converted to the second resolution, and the area important as the image information is not distorted even when converted to the second resolution as the original area. Since the resolution is converted, it has the following features.

  (1) Protect pixels in the area without distorting the image in the original area, and store the image outside the original area in the distorted area by performing reduction processing in each direction. Thus, image information outside the cropping area that was missing in the conventional cropping process can be reflected in the cropping area.

  (2) In the conventional conversion from HD resolution to SD resolution by squeeze processing, the entire converted image is distorted, and thus displaying the SD resolution as it is visually causes a problem. Since the important part can be stored in the SD resolution image area without being distorted, the visual influence can be reduced even when the SD resolution image is displayed as it is.

  (3) Since the entire converted image is distorted in the conversion from HD resolution to SD resolution by the conventional squeeze processing, in order to display the SD resolution image, enlargement processing is performed to cancel the squeeze again. Although it was necessary to display the display area after performing the cropping process again later, in the present invention, since the enlargement process only needs to be performed on the distortion area included in the cropping area to be displayed, the process up to the display is performed. Can be reduced.

  (4) In the present invention, since the original area can be freely set as necessary, the area where the image quality is desired to be maintained can be freely set.

  (5) In the present invention, since the reduction ratio at the time of storing in the distortion area can be changed, it is possible to store the frequency component information of the important part as image information in the distortion area as much as possible as necessary.

  Next, embodiments of the present invention will be described with reference to the drawings. In the present invention, when performing spatial resolution conversion from HD resolution to SD resolution, an original area, which is an area for maintaining original image information as much as possible in the HD resolution, is included in the SD resolution image area to be converted. In addition, while providing some sacrifice of the original image information in the HD resolution, a distortion area, which is an area for capturing the characteristics of the image information in the HD resolution, is included in the SD resolution as much as possible outside the original area and in the SD resolution. Are provided in the image area.

FIG. 1 is an explanatory diagram of an image area converted by an embodiment of the spatial resolution conversion method according to the present invention. It is assumed that the HD resolution image area is an enhancement area. In Figure 1, to define an enhanced region, the upper left coordinate of the area _{11 E (X E, Y E} ), the lateral size of _{W E,} the longitudinal size and _{H E.} In other words, the area 11 is an HD resolution image area and an enhancement area.

Further, it is assumed that the image area of the SD resolution is a cropping area, and in order to define the cropping area, the upper left coordinates of the area 12 are C (X _C , Y _C ), the horizontal size is W _C , and the vertical direction the size of the the _{H C.} In other words, the area 12 is an SD resolution image area and a cropping area. Further, in order to define the original area, the upper left coordinate of the area 13 is O (X _O , Y _O ), the horizontal size is W _O , and the vertical size is H 2 _O. This original area 13 is in the cropping area 12.

  The original area 13 is an area in which an image is stored in the original aspect ratio without distortion of the HD resolution image in the enhancement area 11. On the other hand, in the cropping area 12 and outside the original area 13, the HD resolution image information of the enhancement area 11 outside the original area 13 is distorted by predetermined filtering and reduction processing. This is an image information area (distortion area). In other words, the area within the cropping area 12 and outside the original area 13 sacrifices some of the original image information within the HD resolution, and the image within the HD resolution is within the cropping area 12 with the SD resolution as much as possible. This is a distortion region for capturing information features.

  In the present invention, the spatial resolution of the HD resolution image information in the enhancement area 11 is converted into the spatial resolution of the image information in the cropping area 12 including the original area 13, or the spatial resolution in the reverse direction is converted. Is.

  FIG. 2 illustrates a process of performing spatial resolution conversion from a general HD resolution of 1920 horizontal pixels and 1080 vertical pixels to an SD resolution of 720 horizontal and 480 vertical pixels according to an embodiment of the spatial resolution conversion method of the present invention. Show. FIG. 3 is a flowchart for explaining processing of an embodiment of the spatial resolution conversion method of the present invention. A processing process according to an embodiment of the present invention will be described below with reference to FIGS.

  First, a high-definition HD resolution image frame is acquired (step S101 in FIG. 3). The entire HD resolution image area 21 shown in FIG. 2A is an enhancement area (corresponding to 11 in FIG. 1). Subsequently, a cropping region (corresponding to 12 in FIG. 1) in the enhancement region 21 is specified (step S102 in FIG. 3). In this step S102, as shown by a dotted line in FIG. 2A, the vertical size of the cropping region is changed to the vertical size of the HD resolution image region by multiplying the SD resolution image region by 2.25 times. Match. In addition, by setting a predetermined position in the horizontal direction of the cropping area, a cropping area 22 indicated by a dotted rectangle in FIG.

  Here, as shown by a dotted rectangle in FIG. 2A, the cropping region 22 is arranged so that the region outside the cropping region is uniform. This corresponds to a case where it is assumed that more important contents are generally included in the vicinity of the center of the screen. However, the arrangement of the cropping area is not particularly limited as long as it is within the enhancement area 21, and it is desirable to arrange the cropping area at a position where more video features can be left when converted to the SD resolution.

  Next, the shape of the original area (corresponding to 13 in FIG. 1) for maintaining the image information is specified (step S103 in FIG. 3), and the original area is set (step S104 in FIG. 3). Here, the shape of the original area is a rectangle, and the original area is set at a position indicated by 23 in FIG. Note that the shape of the original area may be specified by giving a shape mode ID as shown in FIG. The arrangement of the original area 23 is not particularly limited as long as it is within the cropping area 22.

  Next, a distortion area is set (step S105 in FIG. 3). Here, a region that is folded back to the cropping region side by an amount corresponding to a region where information is normally lost outside the cropping region is defined as a distortion region. That is, two regions within the cropping region 22 and outside the original region 23 are set as distortion regions (corresponding to 26 and 27 in FIG. 2C described later). As a result, as shown in FIG. 2B, the image information of the left region 24 outside the original region 23 and the image information of the right region 25 outside the original region 23 are respectively converted into distortion regions (described later). 2 (corresponding to 26 and 27 in FIG. 2C).

  Subsequently, a reduction method is specified for the image information of the regions 24 and 25 to be reduced (step S106 in FIG. 3), and predetermined spatial filtering processing and reduction processing in the horizontal direction are performed (FIG. 3). The result is stored in the distortion region (step S108 in FIG. 3). By performing such processing, as shown in FIG. 2C, a cropping area 22 including an original area 23 and distortion areas 26 and 27 is generated.

  The cropping area 22 shown in FIG. 2 (C) is an area obtained by multiplying the SD resolution area, which is the final target, by 2.25 in both the vertical and horizontal directions, and therefore includes the original area 23 and distortion areas 26 and 27. By performing a process of reducing the image information of the cropping area by 1 / 2.25 times by a predetermined spatial filtering and reduction process, as shown in FIG. 2E, the original area 28 and the distortion areas 31, 32 are obtained. An image having the SD resolution size after the spatial resolution conversion, which is the final target configured by the step S109, can be obtained (step S109 in FIG. 3).

  In the above description, the image information in the areas 24 and 25 outside the original area 23 shown in FIG. 2B has been described as being reduced. However, the entire image information shown in FIG. Predetermined spatial filtering and reduction processing is performed on the image information of the image area, and the image is reduced to 1 / 2.25 times to obtain a reduced image including areas 28, 29, and 30 as shown in FIG. After being obtained, predetermined spatial filtering and reduction processing is performed on the image information of the areas 29 and 30 outside the original area 28 of the reduced image, and the result is stored in the distortion areas 31 and 32, so that FIG. You may make it obtain the image of the size of SD resolution after the spatial resolution conversion which is the final target comprised by the original area | region 28 and the distortion area | regions 31 and 32 shown to (E).

  Further, in each of the above-described embodiments, the two-stage reduction processing is performed for the distortion region. Of the image information of all regions shown in FIG. 2B, the left region outside the original region 23 For the image information of 24 and the image information of the right region 25 outside the original region 23, a predetermined spatial filtering and reduction process in one step is performed simultaneously in the horizontal direction and the vertical direction, and the result is the same. In addition to storing in the distorted areas 31 and 32 shown in FIG. (E), the original area 23 is reduced to 1 / 2.25 times by performing a predetermined spatial filtering and reduction process once. You may make it obtain the original area | region 28 shown to the figure (E).

  By performing such processing, the image quality equivalent to that of the conventional cropping process is maintained for the original area 28, and the distortion of the entire image caused by the conversion from the HD resolution to the SD resolution by the conventional squeeze process is performed. Does not occur in the visually important original area 28, and the visual influence can be reduced. Further, in the above-described embodiment, the image information included in the enhancement region other than the cropping region can be stored with the least deterioration in the distortion regions 31 and 32, and the cropping that has been lost in the conventional cropping process. Image information outside the area can be reflected in the clobbing area.

Next, a reduction method in the spatial resolution conversion method of the present invention will be described. FIG. 5 shows an example of a reduction method for creating a distortion region in the method of the present invention. Here, in order to simplify the explanation, it is assumed that spatial resolution conversion as shown in FIG. 2 is performed. In FIG. 5, the boundary between the original area and the distortion area is the origin, B is the length to the boundary of the cropping area, and A is the length to the boundary of the enhancement area. In the example of FIG. 2, it is assumed that the horizontal lengths of the regions 23 and 24 shown in FIG. 2B are A, and the horizontal lengths of the strain regions 26 and 27 shown in FIG. . Also, in general, as shown in FIG. 1, based on the boundary between the original area 13 and the strained _{region, D EO1, D EO2, D} EO3, D EO4 it corresponds to the above _A, D _{CO1, D CO2} , D _CO3 , D _CO4 correspond to the above B.

  In FIG. 5, 41 is an example of a conversion formula for rearranging the positions of the pixels included in the section A area to the pixels included in the section B area. The domain of the conversion formula 41 is shown in 42. When γ in the conversion formula 41 is “1”, the characteristic conversion indicated by 43 in FIG. 5A and the conversion shown in FIG. As the above γ becomes smaller than “1”, the original area as shown in the characteristic 44 and FIG. 5C shown in FIG. 5A and the characteristic 45 and FIG. The pixels can be rearranged so that the reduction ratio changes as the distance from the origin, which is the boundary between and the distortion region, increases.

  Further, as a technique used in the predetermined spatial filtering, generally, the filter characteristic Lanczos 2 represented by the following formula 1 or the filter characteristic Lanczos 3 represented by the following formula 2 is used.

なお、所定の空間フィルタリングで用いられる手法としては、上記以外の周波数変換に基づく縮小方法、ローパスフィルタを利用した手法であっても構わず、本発明において特に限定されるものではないことに注意する。

Note that the method used in the predetermined spatial filtering may be a reduction method based on frequency conversion other than the above or a method using a low-pass filter, and is not particularly limited in the present invention. .

  Next, the state of the resolution conversion process for the area adjacent to the original area in the embodiment of the spatial resolution conversion method of the present invention will be described. 6A, 6B, 6C, and 6D show the state of resolution conversion processing for the upper, lower, left, and right regions of the original region. An area 52 existing above the original area 13 shown in FIG. 6A is a distortion area. By performing predetermined spatial filtering and reduction processing on the images included in the areas 51 and 52, the distortion area 52 is obtained. To store. That is, as shown by 51 and 52 in FIG. 6A, the upper pixel area outside the original area 13 is subjected to reduction processing by predetermined filtering so as to be within the distortion area 52 and stored in the distortion area 52. To do.

  Further, the lower areas 53 and 54 of the original area 13 shown in FIG. 6B, the areas 55 and 56 on the left side of the original area 13 shown in FIG. 6C, and the original area 13 shown in FIG. By performing predetermined spatial filtering and reduction processing on the right regions 57 and 58 as well, the image information of these regions is similarly stored in the distortion regions 54, 56, and 58. Thereafter, the HD resolution is converted to the SD resolution by performing a reduction process by predetermined filtering in order to obtain the original SD resolution.

  7A, 7B, 7C, and 7D show the state of resolution conversion processing for the upper left, lower left, upper right, and lower right areas of the original area. An area 62 existing on the upper left side of the original area 13 shown in FIG. 7A is a distortion area, and the distortion area is obtained by performing predetermined spatial filtering and reduction processing on the images included in the areas 61 and 62. 62.

  Further, the lower left areas 63 and 64 of the original area 13 shown in FIG. 7B, the upper right areas 65 and 66 of the original area 13 shown in FIG. 7C, and the original area 13 shown in FIG. By performing predetermined spatial filtering and reduction processing on the lower right regions 67 and 68 as well, the image information of these regions is similarly stored in the distortion regions 64, 66 and 68.

  8A, 8B, 8C, and 8D show the state of resolution conversion processing for the upper, lower, left, and right regions of the original region. An area 72 existing above the original area 13 shown in FIG. 8A is a distortion area, and by performing predetermined spatial filtering and reduction processing on the images included in the areas 71 and 72, the distortion area 72 is obtained. To store.

  8B, the lower areas 73 and 74 of the original area 13, the left areas 75 and 76 of the original area 13 shown in FIG. 8C, and the original area 13 shown in FIG. 8D. By performing predetermined spatial filtering and reduction processing on the right regions 77 and 78 as well, image information of these regions is similarly stored in the distortion regions 74, 76, and 78.

  Here, in the process as shown in FIG. 6, since the vertical and horizontal processes are performed separately, the upper left, upper right, lower left, and lower right areas 61 to 68 shown in FIG. By performing spatial filtering and reduction processing, processing different from the region shown in FIG. 8 may be performed. For the areas 71 to 78 shown in FIG. 8, the horizontal and vertical filtering and reduction processes as shown in FIG. 6 are performed. As described above, the resolution conversion processing for the region adjacent to the original region is performed by the filtering and reduction processing of FIG. 6, or the resolution conversion processing for the region adjacent to the original region by combining the filtering and reduction processing of FIG. 7 and FIG. I do.

  Next, a spatial scalable encoding method using the spatial resolution conversion method of the present invention will be described with reference to the image conversion process of FIG. First, in performing spatial scalable encoding, the original region shown in FIG. 2E is obtained by performing the spatial resolution conversion processing of the present invention as shown in FIG. 2 using an image frame having the original HD resolution. 28 and an image frame having a size of SD resolution composed of the distortion regions 31 and 32 is obtained.

  An SD resolution image frame including the original area and the distortion area is used as a base layer, and predetermined encoding is performed on the base layer. The enhancement layer is the image frame 97 having the original HD resolution shown in FIG. Thereafter, by performing predetermined decoding, as shown in FIG. 9A, a decoded image frame having a size of SD resolution composed of the original area 81 and the distortion areas 82 and 83 is obtained. Regions 81, 82, and 83 correspond to regions 28, 31, and 32 in FIG. Note that the hatched areas in FIGS. 9A and 9B virtually represent nonexistent areas that are no longer necessary due to the distortion processing described above, although an image exists in the original HD resolution. It is a thing.

  As shown in FIG. 9B, the entire cropped area of the decoded image frame including the original area 81 and the distortion areas 82 and 83 is enlarged 2.25 times by predetermined spatial filtering and enlargement processing. Then, a cropping area of the image frame composed of the enlarged original area 89 and the distortion areas 90 and 91 is obtained. Thereafter, predetermined spatial filtering and enlargement processing is performed on the distortion regions 90 and 91, and the processing result is stored outside the original region 89, whereby the original region 89 shown in FIG. An HD resolution image frame composed of the areas 92 and 93 in which is eliminated is obtained.

  Instead of the above processing, predetermined spatial filtering and enlargement processing is first performed on the distortion regions 82 and 83 in FIG. 9A, and the processing result is stored outside the original region. An image frame of a clipping region composed of an original region 81 and regions 86 and 87 in which lateral distortion is eliminated as shown in FIG. 3C is acquired, and predetermined spatial filtering and 2.25 are performed on the entire image frame. By performing the double enlargement processing, an HD resolution enlarged image frame composed of the original area 89 and areas 92 and 93 in which distortion is eliminated may be obtained in the same manner.

  Alternatively, instead of the above-described two-stage enlargement processing of the distortion region, predetermined distortion filtering and enlargement processing is first performed on the distortion regions 82 and 83 in FIG. By performing the predetermined spatial filtering and the enlargement process of 2.25 times, an enlarged image frame of HD resolution including the original area 89 and the areas 92 and 93 in which distortion is eliminated is obtained directly as shown in FIG. You may do it.

  Thereafter, predictive coding between layers is performed between the image frame shown in FIG. 9D obtained from the base layer and the enhancement layer 97 shown in FIG. 9F. In FIG. 9F, a region 97 surrounded by a dotted line indicates a cropping region. An encoded output of the enhancement layer is obtained by performing predetermined encoding on the prediction code information 98 between layers shown in FIG. 9E obtained by the prediction encoding between layers. By performing such processing, the spatial scalable coding method of the present invention can be realized.

  Further, as the spatial scalable decoding method of the present invention, by performing the same process as the base layer decoding process performed by the encoding method, the regions 89, 92, and 93 shown in FIG. 9D are obtained. The prediction code information 98 between layers is obtained by performing the decoding process of an enhancement layer. Thereafter, by performing predictive decoding between layers between the base layer and the enhancement layer, the clipping region 97 is decoded, whereby the spatial scalable decoding method of the present invention can be realized.

  Next, an embodiment of the resolution conversion method of the present invention in the enlargement direction from the SD resolution as the base layer to the HD resolution as the enhancement layer will be further described with reference to FIGS. FIG. 10 shows a conceptual diagram of an embodiment of the resolution conversion method of the present invention in the enlargement direction. When the resolution conversion in the enlargement direction is performed from the SD resolution base layer image shown in FIG. 10A to the HD resolution enhancement layer image shown in FIG. 10D, at least the upper left edge E1 and the right edge of the enhancement layer image area 707 The coordinates (XE1, YE1), (XE2, YE2) of the lower end E2, and the coordinates (XE3, YE3), (XE4, YE4) of the upper left end E3 and the lower right end E4 of the original area 706, or 2 of the enhancement layer image Alternative information for acquiring the coordinate information for specifying the two areas 706 and 707 is required. That is, the original area 706 corresponding on the enhancement layer side can be specified by E3 and E4. The surrounding area 707 is an area for storing an image after the distortion area is restored.

  Further, the coordinates (XB1, YB1) and (XB2, YB2) of the upper left end B1 and the lower right end B2 of the area 702 of the base layer image shown in FIG. 10A, the upper left end B3 and the lower right end B4 of the original area 701 are displayed. The coordinate information (XB3, YB3), (XB4, YB4) or alternative information for acquiring these coordinate information for specifying the two regions 701, 702 of the base layer image is required. That is, the original region 701 on the base layer side can be specified by B3 and B4. The surrounding area 702 is a distortion area.

  Note that the coordinate reference point is described with E1 and B1 as (0, 0) for the sake of simplicity. B2, B3, and B4 are diagrams based on B1. Next, the spatial resolution conversion operation of the present embodiment will be described with reference to the flowchart of FIG. First, by acquiring information for specifying the shape of the original region of the SD resolution base layer image, the shape of the original region (701 in FIG. 10A) is specified (step S201), followed by HD resolution. The coordinate information of E1, E2, E3, E4, and B1, B2, B3, B4 obtained when converting from SD to SD resolution is acquired (step S202). Alternatively, alternative information for acquiring coordinate information for specifying each region included in the enhancement layer and the base layer is acquired, and these coordinates are specified. The original area is specified from these pieces of information (step S203).

  Next, a cropping area is specified (step S204). In the process of step S204, a virtual image M shown in FIG. 10B is considered in order to simplify the description. An area that can be specified by the upper left end M1 and the lower right end M2 of the virtual image M is a corresponding cropping area in the enhancement layer image. Therefore, how to determine the area that can be specified by M1 and M2 will be described below.

  First, attention is paid to points B3 and B4 of the original region 701 of the base layer image and points E3 and E4 of the original region of the enhancement layer image. An enlargement ratio for enlarging the original area 701 of the base layer image to the corresponding original area 706 of the enhancement layer image is calculated from the coordinates of these points. The enlargement ratio is calculated from the ratio between corresponding sides of each region obtained from the points B3, B4 and E3, E4.

  When the base layer image shown in FIG. 10A is enlarged based on the obtained enlargement ratio, a base layer image including an enlarged original area like the virtual image M shown in FIG. 10B is obtained. By the enlargement process, B1 corresponds to M1, B2 corresponds to M2, B3 corresponds to M3, and B4 corresponds to M4.

  However, at this stage, it is not known which point in the enhancement layer image in FIG. 10D corresponds to the points M1, M2, M3, and M4 in FIG. The coordinates of the points M1, M2, M3, and M4 of the virtual image M are translated to correspond so that the coordinates of E4 correspond. By performing such processing, it is specified which position in the enhancement layer image M1 and M2 correspond to. The coordinates of M1 and M2 corrected in this way are specified as corresponding cropping regions in the enhancement layer image.

  Next, a distortion region within the cropping region and outside the original region is specified (step S205). Here, the distortion region 707 is specified from the relationship between E1, E2, E3, and E4 in FIG. In addition, when it is necessary to restore the distortion area from the base layer image first, the enlargement ratio when the virtual image M is created with the size of the enhancement layer image specified from the coordinates of E1, E2, E3, and E4. The virtual image N shown in FIG. 10C is generated by calculating the reduction ratio by the reciprocal of and reducing each coordinate of the enhancement layer image using the reduction ratio.

  An area outside the original area specified by the points N3 and N4 in the area composed of the upper left end N1 and the lower right end N2 of the generated virtual image N can be specified. For these areas, the distortion can be restored by using the resolution conversion method of the present embodiment.

  Next, an enlargement method is specified by acquiring information for specifying an enlargement method for enlarging an image included in the distortion region (step S206), and the distortion region is restored by the specified enlargement method. Is enlarged (step S207). The image after restoring the distortion area is stored outside the original area (step S208). Thereby, for example, the image information obtained by restoring the image information of the distortion area to the original area 705 of the original area 704 of the virtual image N is stored.

  Finally, the image included in the cropping area is enlarged by a predetermined enlargement method (step S209). Here, when the virtual image M is assumed, the cropping area may be enlarged in the cropping area specifying step in step S204. Further, the enlargement process may be performed on the original area in the base layer image which is an image included in the cropping area. When the virtual image N is assumed, after performing the distortion region enlargement step S207 in order to create the virtual image N from the base layer image, the reduced cropping specified from the coordinates of N1 and N2 of the virtual image N The enlargement process in step S208 is performed on the region. In the enlargement at this time, the enlargement ratio obtained from the relationship between E3, E4, B3, and B4 is used.

  In the above embodiment, each region used in the present invention is described with reference to FIG. 1 in order to simplify the description. In this explanation, the original area is assumed to be rectangular, but the original area is not necessarily rectangular. 12 to 17 are conceptual diagrams showing the shape of the original area.

FIG. 12 is an explanatory view of an image area converted by another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example in the case of adopting a rectangle that allows rotation as the original area 1003. In the rectangle of the original area 13 shown in FIG. 1, the upper left coordinates O (X ₀ , Y ₀ ), the horizontal size W _O , and the vertical size H _O are used to define the rectangle. Therefore, it is difficult to define a region in which a rectangle is rotated with respect to the original region 13.

Therefore, in the embodiment of FIG. 12, the rectangular center coordinates O (X ₀ , Y ₀ ) of the original area 1003 and the upper left coordinates S1 (Xs1, Ys1) of the rectangle are defined in the coordinate format. Here, the center coordinate is O, and the auxiliary coordinate is S1. In order to define the enhanced region 12, upper left coordinates of the _E region 1001 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} Assume that the SD resolution image area is a cropping area, and in order to define the cropping area, the upper left coordinate of the area 1002 is C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

FIG. 13 is an explanatory view of an image area converted by another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example in the case where a circle is adopted as the original area 1103. In this case as well, the center coordinate O and auxiliary coordinates are set to S1. Further, in order to define the enhanced region 13, upper left coordinates of the _E region 1101 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} Assume that the SD resolution image area is a cropping area, and in order to define the cropping area, the upper left coordinates of the area 1102 are C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

FIG. 14 is an explanatory view of an image area converted by still another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example in which a rhombus that allows rotation is adopted as the original region 1203. In the case of a rhombus, since the lengths of the diagonal lines are different in the vertical direction and the horizontal direction, center coordinates O, auxiliary coordinates S1, and auxiliary coordinates S2 are defined. Further, in FIG. 14, to define an enhanced area, upper left coordinates of the _E region 1201 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} Assume that the SD resolution image area is a cropping area, and in order to define the cropping area, the upper left coordinates of the area 1202 are C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

  FIG. 15 is an explanatory diagram of an image area converted by still another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example in the case of adopting an ellipse that allows rotation as the original region 1303. The ellipse of the original area 1303 requires two focal points, but the central coordinate O is adopted to maintain consistency with other shapes, and the central coordinate O and the auxiliary coordinate are set with the first focal point coordinate as the auxiliary coordinate S1. Two focal points are specified by calculating another focal point coordinate from S1. Further, by defining the auxiliary coordinates S2 on the ellipse, it is possible to define an ellipse that allows rotation.

Further, in FIG. 15, to define an enhanced area, upper left coordinates of the _E region 1301 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} The SD resolution image area is assumed to be a cropping area, and in order to define the cropping area, the upper left coordinates of the area 1302 are C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

  FIG. 16 is an explanatory diagram of an image area converted by still another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example in the case of adopting a polygon that allows rotation as the original area 1403. By defining the center coordinate O and the auxiliary coordinate S1 of the original area 1403, a circle is first assumed, and the auxiliary coordinate S2 is defined at a distance equally dividing the circumference, thereby defining a polygon that allows rotation. Can do.

Further, in FIG. 16, to define an enhanced area, upper left coordinates of the _E region 1401 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} Assume that the SD resolution image area is a cropping area, and in order to define the cropping area, the upper left coordinate of the area 1402 is C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

  FIG. 17 is an explanatory view of an image area converted by still another embodiment of the spatial resolution conversion method according to the present invention. This embodiment is an example when an arbitrary shape is adopted as the original region 1503. Since the original area 1503 has an arbitrary shape, the arbitrary shape is defined by preparing the center coordinate O and a plurality of auxiliary coordinates S1, S2, and the like.

  However, it is difficult to determine the shapes of the various original areas as described above from the position and number of the center coordinates and auxiliary coordinates. Therefore, in order to specify the shape of the original area, it is preferable to manage by using a correspondence table as shown in FIG. This correspondence table numbers shapes by shape mode ID. In addition, association with information for specifying the number of center coordinates and auxiliary coordinates necessary to express each shape is performed.

  By transmitting such shape mode ID, center coordinate, and auxiliary coordinate information together with the image frame after resolution conversion of the present invention, it is possible to specify the original region, the distortion region, or the like when restoring the distortion region. It is possible to return to the original image frame size correctly. In addition, the spatial filtering and reduction processing when storing in the distortion region may be performed with respect to the normal direction of the original region as shown in FIG.

In FIG. 17, to define an enhanced area, upper left coordinates of the _E region 1501 (X _{E, Y} E), the lateral size of _{W E,} the longitudinal size and _{H E.} Assume that the SD resolution image area is a cropping area, and in order to define the cropping area, the upper left coordinate of the area 1502 is C (X _C , Y _C ), the horizontal size is W _C , and the vertical size is large. Let _HC be.

  In addition, this invention is not limited to said embodiment, The computer program which makes the computer execute the spatial resolution conversion method of said embodiment is also included. Here, the above computer program is recorded on a recording medium, and may be taken into the computer from the recording medium, or may be distributed via a communication network and downloaded to the computer.

  The present invention can be applied to a resolution conversion apparatus, a resolution conversion program, and the like. Further, the present invention can be applied to a resolution conversion unit of a scalable encoding device, a resolution conversion unit of a scalable encoding program, and the like. Furthermore, the present invention can be applied to a resolution conversion unit of a scalable decoding device, a resolution conversion unit of a scalable decoding program, and the like.

It is explanatory drawing of the image area | region converted by one Embodiment of the spatial resolution conversion method of this invention. It is a conceptual diagram for showing the process of the resolution conversion of one embodiment of the method of the present invention. It is a flowchart for the resolution conversion process description of one embodiment of the method of the present invention. It is a figure for showing the number of coordinates required in order to express the kind and shape of shape of a distortion field of the present invention. It is a conceptual diagram explaining an example of the reduction method for creating the distortion area | region in this invention method. It is a conceptual diagram for showing the state of the resolution conversion processing for the upper, lower, left, and right areas of the resolution conversion of the present invention. It is a conceptual diagram for showing the state of the resolution conversion processing for the upper left, upper right, lower left, and lower right regions of the resolution conversion of the present invention. In the resolution conversion of this invention, it is a conceptual diagram for demonstrating the mode of the resolution conversion process with respect to the area | region of the upper side, the lower side, the left side, and the right side except the area | region of the upper left side, upper right side, lower left side, and lower right side. It is the conceptual diagram (the 1) for showing the process of the spatial scalable encoding using the resolution conversion process of this invention. It is the conceptual diagram (the 2) for showing the process of the spatial scalable encoding using the resolution conversion process of this invention. It is a flowchart for the resolution conversion process description of other embodiment of the method of this invention. It is a conceptual diagram for showing a situation when the shape of the distortion region when performing the resolution conversion processing of the present invention is a rectangle that allows rotation. It is a conceptual diagram for showing a mode when the shape of the distortion area | region at the time of performing the resolution conversion process of this invention is made circular. It is a conceptual diagram for showing a situation when the shape of the distortion region when performing the resolution conversion processing of the present invention is a rhombus that allows rotation. It is a conceptual diagram for showing a situation when the shape of the distortion region when performing the resolution conversion processing of the present invention is an ellipse that allows rotation. It is a conceptual diagram for showing a mode when the shape of the distortion area | region at the time of performing the resolution conversion process of this invention is made into the polygon which accept | permits rotation. It is a conceptual diagram for showing a mode when the shape of the distortion area | region at the time of performing the resolution conversion process of this invention is made into arbitrary shapes. It is a conceptual diagram for showing the mode of the resolution conversion performed by the conventional scalable encoding. It is a conceptual diagram for showing the state of conventional cropping processing and letterbox processing. It is the conceptual diagram (the 1) for showing the process of the spatial scalable encoding using the conventional cropping process. It is a conceptual diagram (the 2) for showing the process of the spatial scalable encoding using the conventional cropping process. It is a conceptual diagram for showing the process of the spatial scalable encoding using the conventional letterbox process. It is a conceptual diagram for showing the state of the conventional squeeze process. It is a conceptual diagram for showing the process of spatial scalable encoding using the conventional squeeze processing.

Explanation of symbols

11, 21 Enhanced region 12, 22, 97 Cropping region 13, 23, 28, 81, 89, 701, 706 Original region 26, 27, 31, 32, 82, 83, 90, 91, 702 Distorted region

Claims (6)

Spatial resolution conversion for converting a video signal having a first spatial resolution into a video signal having a second spatial resolution lower than the first spatial resolution in units of image frames constituting the video signal. A method,
An image region for forming a second image frame that is an image frame of the video signal having the second spatial resolution within a first image frame that is an image frame of the video signal having the first spatial resolution. A first step of identifying a cropping region;
Information for specifying the first original area in the first image frame while specifying the first original area that is an area for performing spatial resolution conversion without distorting the image shape in the cropping area. A second step of generating certain first region information;
A third step of identifying an area outside the first original area in the cropping area as a first distortion area that is an area for performing spatial resolution conversion by distorting an image shape;
In accordance with the ratio between the size of the second image frame and the size of the first image frame, the first original area is reduced by the first reduction method without distorting the image shape, and the second with the original region, the said first original region outside the region in the first image frame, so that said first strained region an area that matches the reduced area in the first reduction mode In addition, the second area which is information for specifying the second original area in the second image frame by distorting the image shape by the second reduction method to reduce the second distortion area. And a fourth step of generating information and generating a second image frame having the second spatial resolution by using the second original area and the second distortion area. Spatial resolution conversion Method. A second resolution having a second spatial resolution lower than the first spatial resolution converted from the first image frame of the video signal having the first spatial resolution by the spatial resolution conversion method according to claim 1. the video signal of the image frame, an image frame constituting the video signal, a re-spatial resolution converting method of converting a video signal having the first spatial resolution,
The second image frame to be subjected to spatial resolution conversion is a first image which performs spatial resolution conversion in a first cropping area which is an image area for making the second image frame in the first image frame. A second area obtained by reducing the original area without distorting the image shape by the first reduction method according to the ratio between the size of the second image frame and the size of the first image frame. The original area and the area outside the first original area in the first image frame are made to coincide with the area obtained by reducing the first distortion area by the first reduction method. A second distortion region obtained by distorting and reducing the image shape by the second reduction method;
A first step of specifying the second original area in the second image frame based on second area information that is information for specifying the second original area in the second image frame. When,
A second step of identifying the second image frame as a second cropping region;
A third step of identifying an area outside the second original area in the second cropping area as the second distortion area;
Depending on the ratio of the size of the second image frame and the size of the first image frame, the second original area is enlarged without distorting the image shape by the first enlargement method. 1 original area is restored, each size of the first and second image frames, and first area information which is information for specifying the first original area in the first image frame; based on said second area information, the second strained region, such that said first said in the image frame of the first original area outside the realm match area, the second expansion method The image shape is distorted and enlarged to restore the area outside the first original area, and the restored first original area and the restored first original area are used to restore the first original area . Spatial resolution The fourth step and the spatial resolution converting method characterized by comprising a restoring the first image frame. Spatial resolution conversion for converting a video signal having a first spatial resolution into a video signal having a second spatial resolution lower than the first spatial resolution in units of image frames constituting the video signal. Is a spatial resolution conversion program for causing a computer to execute
In the computer,
An image region for forming a second image frame that is an image frame of the video signal having the second spatial resolution within a first image frame that is an image frame of the video signal having the first spatial resolution. A first step of identifying a cropping region;
Information for specifying the first original area in the first image frame while specifying the first original area that is an area for performing spatial resolution conversion without distorting the image shape in the cropping area. A second step of generating certain first region information;
A third step of identifying an area outside the first original area in the cropping area as a first distortion area that is an area for performing spatial resolution conversion by distorting an image shape;
In accordance with the ratio between the size of the second image frame and the size of the first image frame, the first original area is reduced by the first reduction method without distorting the image shape, and the second with the original region, and the region where the realm outside the first original region in the first image frame coincides with the first strained region said first reduced area in reduction mode As described above, the second reduction method is used to specify the second original area in the second image frame by distorting the image shape by the second reduction method and reducing it to the second distortion area. Generating region information, and sequentially executing a fourth step of generating a second image frame having the second spatial resolution by the second original region and the second distortion region. Characteristic spatial solution Image conversion program. A second resolution having a second spatial resolution lower than the first spatial resolution converted from the first image frame of the video signal having the first spatial resolution by the spatial resolution conversion method according to claim 1. the video signal of the image frame, an image frame constituting the video signal, again, the spatial resolution conversion for converting a video signal having a first spatial resolution, spatial resolution conversion program for causing a computer to execute Because
The second image frame to be subjected to spatial resolution conversion is a first image which performs spatial resolution conversion in a first cropping area which is an image area for making the second image frame in the first image frame. A second area obtained by reducing the original area without distorting the image shape by the first reduction method according to the ratio between the size of the second image frame and the size of the first image frame. The original area and the area outside the first original area in the first image frame are made to coincide with the area obtained by reducing the first distortion area by the first reduction method. A second distortion region obtained by distorting and reducing the image shape by the second reduction method;
A first step of specifying the second original area in the second image frame based on second area information that is information for specifying the second original area in the second image frame. When,
A second step of identifying the second image frame as a second cropping region;
A third step of identifying an area outside the second original area in the second cropping area as the second distortion area;
Depending on the ratio of the size of the second image frame and the size of the first image frame, the second original area is enlarged without distorting the image shape by the first enlargement method. 1 original area is restored, each size of the first and second image frames, and first area information which is information for specifying the first original area in the first image frame; based on said second area information, the second strained region, such that said first said in the image frame of the first original area outside the realm match area, the second expansion method The image shape is distorted and enlarged to restore the area outside the first original area, and the restored first original area and the restored first original area are used to restore the first original area . Spatial resolution Spatial resolution conversion program to the fourth feature that is sequentially executed and restoring the first image frame. Spatial resolution conversion for converting a video signal having a first spatial resolution into a video signal having a second spatial resolution lower than the first spatial resolution in units of image frames constituting the video signal. A device,
  An image region for forming a second image frame that is an image frame of the video signal having the second spatial resolution within a first image frame that is an image frame of the video signal having the first spatial resolution. Cropping area specifying means for specifying a cropping area;
  Information for specifying the first original area in the first image frame while specifying the first original area that is an area for performing spatial resolution conversion without distorting the image shape in the cropping area. Area specifying / area information generating means for generating certain first area information;
  First distortion area specifying means for specifying an area outside the first original area in the cropping area as a first distortion area that is an area for performing spatial resolution conversion by distorting an image shape;
  In accordance with the ratio between the size of the second image frame and the size of the first image frame, the first original area is reduced by the first reduction method without distorting the image shape, and the second And an area outside the first original area in the first image frame is an area that coincides with an area obtained by reducing the first distortion area by the first reduction method. In addition, the second area which is information for specifying the second original area in the second image frame by distorting the image shape by the second reduction method to reduce the second distortion area. Second image frame generating means for generating information and generating a second image frame having the second spatial resolution by the second original area and the second distortion area;
  A spatial resolution conversion device characterized by comprising: A video signal of a second image frame having a second spatial resolution lower than the first spatial resolution converted from the first image frame of the video signal having the first spatial resolution is converted into the video signal. A spatial resolution conversion device for converting again into a video signal having the first spatial resolution in units of configured image frames,
  The second image frame to be subjected to spatial resolution conversion is a first image which performs spatial resolution conversion in a first cropping area which is an image area for making the second image frame in the first image frame. A second area obtained by reducing the original area without distorting the image shape by the first reduction method according to the ratio between the size of the second image frame and the size of the first image frame. The original area and the area outside the first original area in the first image frame are made to coincide with the area obtained by reducing the first distortion area by the first reduction method. A second distortion region obtained by distorting and reducing the image shape by the second reduction method;
  A second original that specifies the second original area in the second image frame based on second area information that is information for specifying the second original area in the second image frame. Area identification means;
  Second cropping area specifying means for specifying the second image frame as a second cropping area;
  A second strain region specifying means for specifying a region outside the second original region in the second cropping region as the second strain region;
  Depending on the ratio of the size of the second image frame and the size of the first image frame, the second original area is enlarged without distorting the image shape by the first enlargement method. 1 original area is restored, each size of the first and second image frames, and first area information which is information for specifying the first original area in the first image frame; Based on the second region information, the second distortion method is performed so that the second distortion region is a region that coincides with a region outside the first original region in the first image frame. An image shape is distorted and enlarged to restore an area outside the first original area, and the first space is obtained by the restored first original area and the restored outside original first area. Resolution First image frame restoring means for restoring the first image frame and
  A spatial resolution conversion device characterized by comprising:

Images