JP5754635B2 - Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, and image decoding program - Google Patents

Description

  The present invention relates to an image encoding device, an image decoding device, an image encoding method, an image decoding method, an image encoding program, and an image decoding program that enable a scalable image aspect ratio change.

  With the diversification of video viewing environments, there is a need for a video compression and transmission method with scalability that allows viewing videos without stress on displays of different sizes and displays with different screen aspect ratios. The concept of compression coding that can be applied to various networks and terminals has been studied in the framework of scalable coding. H.264 / AVC is already part of the standard, and "spatial scalability" that can flexibly respond to image resolution, "temporal scalability" that gives flexibility to the frame rate of moving images, and flexibility in image quality “SNR scalability” is defined.

  However, these scalability cannot be dealt with when the aspect ratio of the screen is changed. Even when the aspect ratio of the screen does not change, the size of the region of interest in the image decreases proportionally as the screen size decreases, and viewing becomes difficult when the screen size exceeds a certain size. Seam carving is a non-linear resizing method that supports display on displays with different screen aspect ratios (see Non-Patent Document 1, for example). Seam carving is a technique that can freely transform the screen aspect ratio while maintaining important areas in the image, and is used in the field of image editing.

  In order to apply seam carving to scalable coding, data for reconstructing an image with an arbitrary aspect ratio during playback must be realized with as little information as possible, that is, with a high compression ratio and as high a quality as possible during playback. It is important to be able to obtain FIG. 9 shows a general configuration of an encoder and decoder for screen aspect ratio / scalable coding using seam carving. In the encoder 3, the final nonlinear reduced image obtained by performing the seam carving on the original image by the seam carving processing unit 31, the pixel value information of the deleted seam, and the position information of the deleted seam. The information is scalable encoded by the compression encoding units 32, 33, and 34. A non-linear reduced image is set as a basic layer, and position information and pixel value information of a deletion seam are set as an enhancement layer (enhancement layer).

  The decoder 4 decodes all the non-linear reduced images that are the basic layer by the decoding unit 41, but the deletion seam information is obtained by using only the partial seams necessary for obtaining the desired screen aspect ratio by the partial decoding units 42 and 43. Decoding and synthesizing unit 44 inserts a necessary number of seams into the image of the basic layer until an image having a target aspect ratio is obtained.

  The most basic method uses both deleted seam position information and pixel value information, and arranges the deleted seam pixels at appropriate locations based on the position information (see, for example, Non-Patent Document 2). ). In this method, since the deleted pixel can be inserted at the correct position, a high-quality decoded image can be obtained. However, since the deletion seam position information and pixel value information are transmitted, a large amount of data is transmitted.

  The method disclosed in Non-Patent Document 1 inserts a seam without using seam position information or pixel value information. In this method, a position where a seam is inserted is estimated using an image in the base layer, and a pixel value to be inserted is interpolated from pixel values in the vicinity of the insertion position. This means that the amount of data to be transmitted is small, but the quality of the restored image is poor and the shape is often unnatural.

  The method disclosed in Non-Patent Document 3 is a method that uses only the position information of the deleted seam. In this method, the position of the pixel to be interpolated can be accurately known. The pixel is inserted by interpolating the pixel value at that position with the average value of the left and right pixels. With this method, the seam insertion position is correct and the image can be restored to a natural shape, but the image quality of the restored image deteriorates in an image where the seam positions are dense.

S.Avian, A.Shamir, “Seam Carving for Content-Aware Image Resizing,” ACM SIGGRAPH 2007, ACM Transactions on Graphics, Vol.26, No3, Article 10, Jul., 2007. Nguyen Thi Nhat Anh, Wenxian Yang, Jianfei Cai, “Seam Carving Extention: a Compression Perspective,” Proceedings of the seventeen ACM international conference on Multimedia (ACM-MM’09). Terada, Inazumi, Hotta, “Study on Spatial Scalable Coding Based on Seam Carving,” PCSJ / IMPS 2009, P-2-02.

  As described above, the conventionally proposed method based on seam carving corresponding to the change in the screen aspect ratio is not sufficient from the viewpoint of image quality and compression rate. Conventional screen aspect ratio / scalable coding using seam carving requires both deleted seam position information and pixel value information in order to restore high-quality images, which increases the amount of transmitted data. is there.

  The present invention has been made in view of such circumstances, and an image code that can reduce the amount of data required while maintaining a high-quality restored image in screen aspect ratio / scalable coding based on seam carving. It is an object to provide an encoding device, an image decoding device, an image encoding method, an image decoding method, an image encoding program, and an image decoding program.

  The present invention is an image encoding apparatus that compresses and encodes an image to be encoded, and performs seam carving processing on the image to be encoded, thereby deleting pixel information of the deleted seam and deletion of the deleted seam. A seam carving processing unit that generates and outputs information indicating the order and reduced image information after the seam is deleted, and compression-codes the pixel information of the deleted seam and the information indicating the deletion order of the deleted seam First encoding means for outputting converted first encoded data, and second encoding means for outputting second encoded data obtained by compression-encoding the reduced image information after the seam is deleted It is characterized by comprising.

  The present invention is an image decoding apparatus that decodes the first encoded data and the second encoded data encoded by the image encoding apparatus to restore an encoding target image, wherein After decoding the encoded data to obtain pixel information of the deleted seam and information indicating the deletion order of the deleted seam, and after decoding the second encoded data and deleting the seam The second decoding means for obtaining the reduced image information of the image and the cost function when the corresponding pixel of the seam is inserted at each pixel position of the reduced image after the seam is deleted is regarded as the image data, and the seam carving process is applied. The seam position to be deleted is obtained, the obtained position is estimated as the position where the seam is to be inserted based on the pixel information of the seam deleted on the reduced image, and the seam is inserted at the estimated position to thereby calculate the code. And further comprising a synthesizing means for restoring the target image.

  In the present invention, the synthesis unit restores an image having an arbitrary aspect ratio by estimating the insertion position of the seam in the reduced image after deleting the seam based on the deletion order of the deleted seam. And

  The present invention is an image encoding method for compressing and encoding an image to be encoded, and by performing a seam carving process on the image to be encoded, the pixel information of the deleted seam and the deletion of the deleted seam A seam carving process step of generating and outputting information indicating the order and reduced image information after deleting the seam, and compressing code for pixel information of the deleted seam and information indicating the deletion order of the deleted seam First encoding step for outputting converted first encoded data, and second encoding step for outputting second encoded data obtained by compression-encoding the reduced image information after deleting the seam It is characterized by having.

  The present invention is an image decoding method for decoding an image to be encoded by decoding the first encoded data and the second encoded data encoded by the image encoding method, A first decoding step of decoding encoded data to obtain pixel information of the deleted seam and information indicating the deletion order of the deleted seam; and after decoding the second encoded data and deleting the seam The second decoding step for obtaining the reduced image information of the image and the cost function when the corresponding pixel of the seam is inserted at each pixel position of the reduced image after the seam is deleted is regarded as the image data, and the seam carving process is applied. And determining the seam position to be deleted, estimating the determined position as a position where the seam should be inserted based on the pixel information of the seam deleted on the reduced image, and inserting the seam at the estimated position And having a synthetic steps to restore the serial coded image.

  According to the present invention, the composition step restores an image having an arbitrary aspect ratio by estimating the insertion position of the seam in the reduced image after deleting the seam based on the deletion order of the deleted seam. And

  The present invention is an image encoding program for compressing and encoding an image to be encoded, and by performing seam carving processing on the image to be encoded, the pixel information of the deleted seam and the deletion of the deleted seam A seam carving process step of generating and outputting information indicating the order and reduced image information after deleting the seam, and compressing code for pixel information of the deleted seam and information indicating the deletion order of the deleted seam First encoding step for outputting converted first encoded data, and second encoding step for outputting second encoded data obtained by compression-encoding the reduced image information after deleting the seam It is characterized by having a computer perform.

  The present invention is an image decoding program for decoding the first encoded data and the second encoded data encoded by the image encoding program to restore an encoding target image, wherein A first decoding step of decoding encoded data to obtain pixel information of the deleted seam and information indicating the deletion order of the deleted seam; and after decoding the second encoded data and deleting the seam The second decoding step for obtaining the reduced image information of the image and the cost function when the corresponding pixel of the seam is inserted at each pixel position of the reduced image after the seam is deleted is regarded as the image data, and the seam carving process is applied. Then, the seam position to be deleted is obtained, the obtained position is estimated as the position where the seam should be inserted based on the pixel information of the seam deleted on the reduced image, and the seam is inserted at the estimated position. Characterized in that to perform a synthesizing step of restoring the encoded target image to a computer by.

  According to the present invention, the composition step restores an image having an arbitrary aspect ratio by estimating the insertion position of the seam in the reduced image after deleting the seam based on the deletion order of the deleted seam. And

  According to the present invention, it is possible to accurately estimate the insertion position of a seam without using seam position information by regarding the cost function of each position when it is assumed that the seam is inserted in the image as an image. Therefore, seam position information is not required as encoded image data with a scalable screen aspect ratio, and it is possible to obtain an arbitrary high-quality aspect ratio, while reducing the amount of encoded data than before. The effect of being able to be obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is a flowchart which shows operation | movement of the apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the apparatus shown in FIG. It is a block diagram which shows the structure of the encoding apparatus and decoding apparatus by a prior art.

  Hereinafter, an image encoding device and an image decoding device according to an embodiment of the present invention will be described with reference to the drawings. First, with reference to FIGS. 4 to 8, a method for estimating a seam insertion position using non-linear reduced image information and deleted seam pixel value information will be described. First, the position where the seam pixel is inserted is calculated on the decoding side without using the position information of the deleted seam. As shown in FIG. 5, the pixel value information of a given seam is simply one-dimensionally arranged pixel values of one seam (G (j)), and the pixel position is unknown. Therefore, using the constraint that the seams shown in FIG. 6 are eight-connected on the original image F (i, j), all the seam patterns that can be generated by eight-links on the image to which the seam is to be inserted. Of these, the most natural one is determined (see FIGS. 7 and 8).

  The seam position estimation method will be specifically described below. Here, the case of a vertical seam will be described as an example, but the present invention can also be applied to insertion of a horizontal seam by switching the vertical and horizontal directions. The seam position estimation method is obtained by the processing operation shown in FIG. Where G (j) is the value of the pixel on the seam (j = 0, 1, 2,... N−1), and F (i, j) is the target image from which the seam is to be inserted. , J, F (i, j) is the pixel value at position (i, j), i = 0, 1, 2,..., M−1, j = 0, 1, 2,. , N−1. N is the length of the seam, which is equal to the number of vertical pixels of the target image. M is the number of horizontal pixels of the target image.

  First, one combination of (i, j) is selected (step S1), and the jth pixel G (j) from the top of the seam is inserted into the pixel position (i, j) of the image F (i, j). The cost of time F ′ (i, J) is calculated (step S2). Then, for all i (i = 0, 1, 2,..., M−1, M), j (j = 0, 1, 2,..., N−1), the cost F ′ (i, j) is repeatedly calculated (step S3). Here, the cost F ′ (i, j) represents how smoothly the portion is connected to the surroundings when the seam pixel G (j) is inserted into the position (i, j) of the image. It is a function that decreases the value.

Three formulas (1), (2), and (3) as cost functions are applicable.
F ′ (i, j) = | GY (j) −FY (i, j) | + | GY (j) −FY (i−1, j) | (1)

F ′ (i, j) = | GY (j) −FY (i, j) | + | GY (j) −FY (i−1, j) | + | GU (j) −FU (i, j) | + | GU (j) −FU (i−1, j) | + | GV (j) −FV (i, j) | + | GV (j) −FV (i−1, j) | (2)

F ′ (i, j) = | GY (j) −FY (i, j) | + | GY (j) −FY (i−1, j) | + | GY (j) −FY (i, j− 1) | + | GY (j) −FY (i−1, j−1) | + | GY (j) −FY (i, j + 1) | + | GY (j) −FY (i−1, j + 1) | (3)

  Here, GY (j) and FY (i, j) are the seam G (j), the value of the luminance signal of the image F (i, j), GU (j), FU (i, j), GV ( j) and FV (i, j) are values of the color difference signal. Further, in order to simplify the description of the cost function when inserting the seam at the left end or the right end of the image, FX (−1, j) = FX (0, j), FX (M, j) = FX (M (1, j)), (1), (2), and (3) are calculated. Here, FX (i, j) is FY (i, j) or FU (i, j) or FV (i, j).

  Next, F ′ (i, j) is regarded as an image, and the same method as when the seam (G (j)) is deleted, that is, the seam (G ( Perform seam carving using the same value as when j)) is deleted to obtain the minimum cost path (step S4). The definition of the minimum cost path by seam carving and how to find it are shown in, for example, Non-Patent Document 1, and are well known, so detailed description is omitted here. Subsequently, the obtained minimum cost path is output as a position where a seam pixel is to be inserted (step S5).

  Next, the configuration of the image encoding device (encoder) and the image decoding device (decoder) according to the embodiment will be described with reference to FIG. The image encoding device and the image decoding device are a scalable encoder and decoder using the seam position estimation described above, and are configured by a computer device. The decoder uses the non-linear reduced image as the basic layer and the pixel value information of the deletion seam as the extended layer to restore the image of any aspect ratio. The encoder uses the non-linear reduced image as the basic layer, and the information as the extended layer is Only the pixel value information of the deletion seam is encoded.

  FIG. 1 is a block diagram showing the configuration of the embodiment. In FIG. 1, reference numeral 1 denotes an encoder that compresses and encodes an original image and outputs the result. Reference numeral 11 denotes a seam carving processing unit that inputs an original image f, performs seam carving processing, and outputs a non-linear reduced image (basic layer) and pixel information (extended layer) of the deleted seam. Reference numeral 12 denotes a compression encoding unit that outputs a base layer F encoded by compressing and encoding a non-linear reduced image. Reference numeral 13 denotes a compression encoding unit that outputs the enhancement layer G encoded by compressing and encoding the pixel information of the deleted seam.

  Reference numeral 2 denotes a decoder that decodes and outputs a compression-encoded image. Reference numeral 21 denotes a decoding unit that receives the encoded base layer F, performs decoding, and outputs a nonlinear reduced image. Reference numeral 22 denotes a partial decoding unit that receives the encoded enhancement layer G, performs decoding, and outputs pixel information of the deleted seam. Reference numeral 23 denotes a synthesis unit that synthesizes and outputs the nonlinear reduced image and the pixel information of the deleted seam.

  Next, the processing operation of the encoder 1 shown in FIG. 1 will be described. Original image f (i, j), i = 0, 1, 2,..., M−1, j = 0, 1,..., N−1 are input images to encoder 1, and the horizontal width is M. The pixel has a height of N pixels. In the image data input to the encoder 1, the seam carving processing unit 11 deletes the M ′ seams and the image has the width M−M ′ pixels (F (i, j)) (i = 0, 1, 2,..., MM′-1, j = 0, 1,. M 'is determined by the minimum size of the displayed nonlinear reduced image. Then, F (i, j) is encoded as basic layer data by the compression encoding unit 12. Since the nonlinear reduced image that is the basic layer is considered to hold statistical properties similar to those of normal image data, the compression method used for normal natural images such as JPEG and JPEG-LS is applied as it is. .

  Further, only the pixel values of the deleted M ′ seams are encoded by the compression encoding unit 13. The pixel value of the j-th pixel from the top of the i-th deleted seam is represented by G (i, j) (i = 0, 1, 2,..., M′−1, j = 0, 1, .., N-1) represents the entire deleted seam. Here, G (i, j) is encoded as data of the enhancement layer by the compression encoding unit 13. Since the pixels adjacent to each other along the seam are pixels in the vicinity of 8 in the original image, it is considered that the correlation is high. Therefore, compression can be performed by applying DPCM or one-dimensional DCT in the order of pixels on the seam.

  Although the vertical seam has been described here, it can also be applied to insertion of a horizontal seam by switching the vertical and horizontal directions. Furthermore, it can be easily expanded when both vertical and horizontal seams are cut. Assume that M 'vertical seams and N' horizontal seams are deleted. In this case, the seam is deleted by determining the order according to the method of Non-Patent Document 1, and the vertical seam is the pixel value of the jth pixel from the top of the vertical seam that has been deleted from the last. G1 (I, j), i = 0, 1, 2,..., M′−1, j = 0, 1,..., N−1, the horizontal seam among the horizontal seams The pixel value of the j-th pixel from the top of the i-th deleted pixel is G2 (i, j), i = 0, 1, 2,..., M−1, j = 0, 1,.・, N'-1. Further, a function α (i), i = 0, 1, 2,..., M ′ + N′−1 for storing the order of seam deletion is generated. If α (i) = 0 or 1, and the i-th deleted seam is a vertical seam, α (i) = 0, and if it is a horizontal seam, α (i) = 1. G1, G2, and α are combined and output from the encoder 1 as pixel information of the deleted seam.

  Next, the processing operation of the decoder 2 shown in FIG. 1 will be described with reference to FIG. The decoder 2 receives the encoded base layer data F (i, j) and the encoded extension layer data G (i, j). The base layer encoded data is decoded by the decoding unit 21 and F (i, j) is restored. This is the seam insertion target image F (step S11).

  On the other hand, in the enhancement layer, the partial decoding unit 22 decodes the necessary seam pixels (step S12). Insert a seam into F (i, j), (i = 0, 1, 2,..., MM′-1, j = 0, 1,..., N−1) In order to obtain an image of horizontal X pixels and vertical N pixels, the last deleted X- (M−M ′) seams are necessary, so G (0, j), G (1, j), ..., the value of G (X-M + M'-1, j) is decoded (step S13). An image is synthesized by inserting X- (M-M ′) seams into the image F of the base layer, and an image of X pixels × N pixels is decoded.

  Next, the synthesizing unit 23 uses the decoded G (0, j) as G (j), and determines the insertion position of G (0, j) in F (i, j) by the processing operation shown in FIG. The estimation is performed (step S14), and G (0, j) is inserted into the estimated position of F (i, j) (step S15). The inserted image is newly set as F (i, j) (step S16). The horizontal width increases by one pixel, i = 0, 1, 2,..., M−M ′. Then, it is determined whether or not the target resolution has been reached (step S17). If not reached, the processing operations of steps S13 to S16 are repeated. Finally, F (i, j), i = 0, 1, 2, ..., the insertion position of G (X-M + M'-1, j) in X-2 is estimated, and G (X-M + M'-1, j) is estimated to F (i, j) Insert in position. As a result, an image of X pixels × N pixels is restored. Then, the decoder 2 outputs this restored image (step S18).

  Next, a modification of the processing operation shown in FIG. 2 will be described with reference to FIG. The processing operation shown in FIG. 3 is different from the processing operation shown in FIG. 2 in that step S12 is deleted and step S13a is provided instead of step S13. The processing operations shown in FIGS. 2 and 3 can be applied to horizontal seam insertion by switching the vertical and horizontal directions. Further, the present invention can be applied as it is when both vertical and horizontal seams are cut. If the seam to be inserted is i-th, a vertical seam is inserted when α (i) = 0, so that the seam deleted at the end of G1 (i, j) that has not been inserted is shown in FIG. The seam G deleted last among the uninserted seams in the extended hierarchy at (step S13a). In addition, since a seam in the horizontal direction is inserted when α (i) = 1, the seam deleted last in G2 (i, j) is not inserted in the extension hierarchy in FIG. The seam G deleted last among the seams.

  In this way, the image coding apparatus which changes the aspect ratio of the image to be scalable, encodes, transmits the encoded data, and uses the transmitted data to restore the image in an arbitrary aspect ratio in a scalable manner, and In the image decoding apparatus, the data required to enable the aspect ratio to be scalable can be reduced more than before, and a higher quality image than before can be restored at the time of restoration.

  As described above, since the position of the seam deleted on the encoding side is estimated based on the pixel value of the seam on the decoding side, the seam position information does not need to be transmitted, and the amount of transmission data is reduced. Can do. Further, since the deleted pixel does not estimate the pixel value but transmits the pixel value itself, the correct image can be restored if the position is known, so that the quality of the restored image can be improved. In addition, for the deletion position estimation, the entire cost function when inserting a seam at each position of the image is regarded as one image, and the deletion position is determined by the same method as the seam deletion from the original image. The estimation accuracy is improved, and the seam deletion position can be estimated with high accuracy.

  Note that a program for realizing the function of the processing unit in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system and executed to encode an image. Processing / decoding processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  In the image encoding process and the image decoding process, it is applicable to a use indispensable for enabling a scalable image aspect ratio change.

  DESCRIPTION OF SYMBOLS 1 ... Encoder (image coding apparatus), 11 ... Seam carving process part, 12, 13 ... Compression coding part, 2 ... Decoder (image decoding apparatus), 21 ... Decoding part, 22 ... partial decoding unit, 23 ... synthesis unit

Claims (6)

An image encoding device that compresses and encodes an image to be encoded,
Value when the i-th deleted seam is a vertical seam to indicate pixel information of the deleted seam and the deletion order of the deleted seam by performing seam carving processing on the encoding target image Is α, a function α (i) having a value of 1 in the case of a horizontal seam (i = 0, 1, 2,..., M ′ + N′−1, M ′ is the number of vertical seams, N ′ is the number of seams in the horizontal direction) and reduced image information after deleting the seams, and seam carving processing means for generating and outputting,
First encoding means for outputting first encoded data obtained by compression encoding the function α (i) indicating pixel information of the deleted seam and the deletion order of the deleted seam;
An image encoding apparatus comprising: second encoding means for outputting second encoded data obtained by compressing and encoding the reduced image information after the seam is deleted. An image decoding apparatus that decodes the first encoded data and the second encoded data encoded by the image encoding apparatus according to claim 1, and restores an encoding target image,
First decoding means for decoding the first encoded data to obtain pixel information of the deleted seam and a function α (i) indicating the deletion order of the deleted seam;
Second decoding means for decoding the second encoded data and obtaining reduced image information after removing the seam;
Based on the function alpha (i), seams seam to be inserted or a longitudinal seam, or while determining whether a next seam, deleted to each pixel position of the reduced image after deleting the seam wherein regarded as image data the cost function was measured with the pixel based on the pixel information, determine the seam location to be deleted in the case of applying the seam carving process on the image data, wherein the obtained seam position reduced image It was estimated as the position to be inserted the sheet over arm above and combining means for restoring the coded image at an arbitrary aspect ratio by inserting the seam based on the pixel information of the seam that removed the estimated position image decoding device comprising a Turkey provided. An image encoding method performed by an image encoding apparatus that compresses and encodes an image to be encoded,
Value when the i-th deleted seam is a vertical seam to indicate pixel information of the deleted seam and the deletion order of the deleted seam by performing seam carving processing on the encoding target image Is α, a function α (i) having a value of 1 in the case of a horizontal seam (i = 0, 1, 2,..., M ′ + N′−1, M ′ is the number of vertical seams, N ′ is the number of seams in the horizontal direction) and the seam carving process step of generating and outputting reduced image information after the seam is deleted;
A first encoding step of outputting first encoded data obtained by compressing and encoding the function α (i) indicating pixel information of the deleted seam and the deletion order of the deleted seam;
And a second encoding step of outputting second encoded data obtained by compressing and encoding the reduced image information after the seam is deleted. An image decoding method performed by an image decoding apparatus that decodes the first encoded data and the second encoded data encoded by the image encoding method according to claim 3 to restore an encoding target image. And
A first decoding step of decoding the first encoded data to obtain pixel information of the deleted seam and a function α (i) indicating the deletion order of the deleted seam;
A second decoding step of decoding the second encoded data to obtain reduced image information after removing the seam;
Based on the function alpha (i), seams seam to be inserted or a longitudinal seam, or while determining whether a next seam, deleted to each pixel position of the reduced image after deleting the seam wherein regarded as image data the cost function was measured with the pixel based on the pixel information, determine the seam location to be deleted in the case of applying the seam carving process on the image data, wherein the obtained seam position reduced image It was estimated as the position to be inserted the sheet over arm above a combining step for restoring the coded image at an arbitrary aspect ratio by inserting the seam based on the pixel information of the seam that removed the estimated position image decoding method comprising Rukoto to Yusuke.   An image encoding program for causing a computer to execute the image encoding method according to claim 3.   An image decoding program for causing a computer to execute the image decoding method according to claim 4.

Images