JP2017060111A - Image encoder, image decoder, image encoding method and image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily confirm an imaging content for a light field image and to achieve highly efficient compression encoding.SOLUTION: An image encoder comprises: light field image parameter setting means for setting a light field image parameter which characterizes a first light field image on the basis of the first light field image; base image generating means for generating a base image being a normal image of the same scene as that of the first light field image by using the first light field image; base image encoding means for encoding the base image; light field image generating means for generating a second light field image to the light field image of an encoding object by using the base image in accordance with the light field image parameter; and light field image encoding means for encoding the first light field image of the encoding object to output a bit stream while a second light field image is set to be a prediction image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image encoding device, an image decoding device, an image encoding method, and an image decoding method for encoding and decoding an image.

  The spatial resolution is a very important factor in the quality of digital images and moving images. Therefore, research and development of a high-definition moving image / image system capable of handling a higher-resolution moving image is continuously performed. By using a high-resolution moving image / image, it is possible to clearly express the subject and the background in detail. On the other hand, when the resolution of whether or not each subject is in focus is low, elements that could not be visually recognized are visible. In general, a moving image / image in which the subject to be watched is out of focus is recognized as blurring, and the image quality is evaluated to be low. For this reason, it is considered that it is very important to accurately control the focus when shooting a high resolution moving image / image.

  Note that in this specification, an image refers to an image for one frame constituting a still image or a moving image.

  However, it is known that focus control when taking an image is a very difficult task. When shooting low-resolution moving images / images, it is possible to shoot while checking the focus status using a viewfinder or small monitor for confirmation, but shooting high-resolution images / videos. In this case, it is because a small monitor cannot confirm a fine focusing state.

  On the other hand, an imaging apparatus that can adjust the focus after shooting has been developed on the assumption that image processing is performed after shooting. Non-Patent Document 1 is an imaging device called a light field camera, which has a configuration in which a microlens array is inserted between a main lens and a projection surface in a conventional camera. By adopting such a configuration, it is possible to record the light rays incident on the camera for each incident angle, and it is possible to generate a moving image / image focused at different distances therefrom. An image captured by a light field camera (hereinafter referred to as a light field image) is an image expressing the intensity of light rays at each pixel position for each traveling direction of the light rays.

R. Ng, "Digital light field photography", Ph.D dissertation, Stanford University, July 2006.

  By using the method described in Non-Patent Document 1, it is possible to perform focus adjustment by performing processing after shooting. However, unlike an ordinary image, an image photographed according to Non-Patent Document 1 records light beams separated for each angle by a microlens. Therefore, when an ordinary image coding method is used, an efficient code is used. There is a problem that it cannot be realized. Furthermore, there is a problem that an image taken according to Non-Patent Document 1 cannot be confirmed like a general image unless image processing is performed.

  The present invention has been made in view of such circumstances, and it is possible to easily confirm the captured content of a light field image captured by a light field camera and to realize highly efficient compression coding. An object is to provide an image encoding device, an image decoding device, an image encoding method, and an image decoding method.

  One aspect of the present invention is an image encoding device that performs encoding of a first light field image obtained by a light field camera, and the first light field image is converted based on the first light field image. A light field image parameter setting means for setting a light field image parameter to be characterized, and a base for generating a base image which is a normal image of the same scene as the first light field image, using the first light field image An image generation unit, a base image encoding unit for encoding the base image, and a second light field image for the encoding target light field image using the base image according to the light field image parameter A light field image generating means and the second laser While the door field image and the prediction image, is an image encoding device and a light field image encoding means for said first light field image to be coded and outputs a bit stream by encoding.

  One aspect of the present invention is the image encoding device, wherein the base image generation unit generates an omnifocal image focused on the entire surface of the first light field image as the base image.

  One aspect of the present invention is the image encoding device, wherein the base image generation means is a sub-aperture that is an image of a microlens located in the center of a main lens of the light field camera in the first light field image. An image is generated as the base image.

  One aspect of the present invention is an image decoding apparatus that decodes the first light field image from a bit stream of the first light field image obtained by a light field camera, wherein the first light field image is Light field image parameter setting means for decoding a parameter to be characterized and setting the decoded parameter as a light field image parameter, and base image decoding means for decoding a base image that is a normal image in the same scene as the first light field image And a light field image generating means for generating a second light field image for the light field image to be encoded using the base image according to the light field image parameters, and the second light field image as a predicted image. While An image decoding apparatus and a light field image decoding means for decoding the first light field image.

  One aspect of the present invention is the image decoding apparatus, wherein the base image decoding unit decodes an omnifocal image focused on the entire image of the same scene as the first light field image as the base image. To do.

  One aspect of the present invention is the image decoding device, wherein the base image decoding unit includes a sub-aperture image that is an image of a microlens positioned in the center of a main lens of the light field camera in the first light field image. Are decoded as the base image.

  One aspect of the present invention is an image encoding method performed by an image encoding device that encodes a first light field image obtained by a light field camera, the first encoding method based on the first light field image. A light field image parameter setting step for setting a light field image parameter characterizing one light field image, and a normal image of the same scene as the first light field image using the first light field image A base image generating step for generating a base image; a base image encoding step for encoding the base image; and a second image for a light field image to be encoded using the base image according to the light field image parameters. Light fee for generating light field images Image coding step, and a light field image coding step of coding the first light field image to be coded and outputting a bitstream while using the second light field image as a predicted image It is a conversion method.

  One aspect of the present invention is an image decoding method performed by an image decoding apparatus that performs decoding of the first light field image from a bit stream of the first light field image obtained by a light field camera. A light field image parameter setting step for decoding parameters that characterize the light field image and setting the decoded parameters as light field image parameters; and decoding a base image that is a normal image of the same scene as the first light field image A base image decoding step, a light field image generation step of generating a second light field image for the light field image to be encoded using the base image according to the light field image parameters, and the second light field image While the field image and the predicted image is an image decoding method and a light field image decoding step of decoding said first light field image.

  According to the present invention, it is possible to easily confirm the captured content, and to obtain an effect that it is possible to simultaneously realize highly efficient compression coding of a light field image.

It is a block diagram which shows the structure of the image coding apparatus in embodiment of this invention. It is a flowchart which shows operation | movement of the image coding apparatus 100 shown in FIG. It is a block diagram which shows the structure of the image decoding apparatus in embodiment of this invention. It is a flowchart which shows operation | movement of the image decoding apparatus 200 shown in FIG. It is a block diagram which shows the hardware constitutions when the image coding apparatus 100 shown in FIG. 1 is comprised by a computer and a software program. It is a block diagram which shows the hardware constitutions when the image decoding apparatus 200 shown in FIG. 3 is comprised by a computer and a software program.

  Hereinafter, an image encoding device and an image decoding device according to an embodiment of the present invention will be described with reference to the drawings. Here, a process for one image will be described, but a moving image (video) can be encoded and decoded by repeating the process for a plurality of consecutive images. Note that the processing according to the present technique may be applied to some frames without being applied to all frames of the moving image, and another processing may be applied to other frames.

  FIG. 1 is a block diagram illustrating a configuration of an image encoding device according to the present embodiment. As shown in FIG. 1, the image encoding apparatus 100 includes a light field image input unit 101, a light field image parameter setting unit 102, a light field image parameter encoding unit 103, a base image generation unit 104, and a base image encoding unit 105. A base image decoding unit 106, a light field image synthesis unit 107, a light field image encoding unit 108, and a multiplexing unit 109.

  The light field image input unit 101 inputs a light field image to be encoded. Hereinafter, this light field image is referred to as an encoding target light field image. Any type of light field image may be input. For example, even a light field image obtained by using a plurality of microlenses, an optical image of a subject formed by a main lens as in Non-Patent Document 1, may be a light field image obtained using another method. It doesn't matter. Here, it is assumed that the light field image of Non-Patent Document 1 is input.

  The light field image parameter setting unit 102 sets parameters that characterize the light field image. Any parameter may be set, but it is assumed that the light field image synthesis unit 107 described later sets parameters necessary when generating the encoding target light field image. For example, the arrangement of microlenses in the microlens array, the rotation / translational displacement between the microlens array and the image sensor, the focus parameter of each microlens, the number of image elements corresponding to the microlens, the RGB arrangement of the image sensor, and encoding There is a depth map for the target light field image. Hereinafter, these parameters are collectively referred to as encoding target light field image parameters. The light field image parameter encoding unit 103 encodes the encoding target light field image parameter.

  The base image generation unit 104 generates an image taken with a general camera from the encoding target image light field. Hereinafter, this image is referred to as a base image. The base image encoding unit 105 encodes the base image. The base image decoding unit 106 generates a base image by decoding a bit stream for the base image. The light field image combining unit 107 generates a combined light field image (second light field image) of the encoding target light field image from the decoded base image according to the light field image parameter. Here, generating an image (combined light field image) from the decoded base image in accordance with the light field image parameter is called combining. The light field image encoding unit 108 encodes the encoding target light field image while using the combined light field image as a predicted image. The multiplexing unit 109 multiplexes and outputs the base image bit stream, the light field image parameter bit stream, and the encoding target light field image bit stream.

  Next, the operation of the image coding apparatus 100 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the image coding apparatus 100 shown in FIG. First, the light field image input unit 101 inputs an encoding target light field image (step S101).

  When the input of the encoding target light field image is completed, the light field image parameter setting unit 102 sets the encoding target light field image parameter for the encoding target light field image (step S102). The encoding target light field image parameter may be input to the image encoding apparatus 100 or may be estimated by analyzing the encoding target light field image. For example, the arrangement of microlenses in the microlens array, the rotation / translational displacement between the microlens array and the image sensor, the focus parameter of each microlens, the number of image elements corresponding to the microlens, the RGB array of the image sensor, etc. Since the information does not depend on the image, it may be input from the outside.

  On the other hand, since the depth map for the encoding target light field image is information depending on the image, it may be generated by analyzing the encoding target light field image. As a method for estimating the depth map from the light field image, any method may be used. For example, Reference 1 “S. Wanner, and B. Goldluecke,“ Globally consistent depth labeling of 4D light fields, "2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.41-48, June 201.2" may be used.

  Next, the light field image parameter encoding unit 103 encodes the set encoding target light field image parameter (step S103). Any method may be used for encoding. For encoding the depth map for the light field image to be encoded, any image encoding method or moving image encoding method may be used.

  When the encoding of the light field image parameter is completed, the base image generation unit 104 generates a base image from the encoding target light field image according to the encoding target light field image parameter (step S104). The base image generated here may be any normal image captured at the same position, orientation, angle of view, etc. as the encoding target light field image. For example, it may be an omnifocal image or an image having a specific depth of field. The base image may be generated by any method. For example, if the image has a specific depth of field, the Fourier slice method (reference document 2 “R. Ng,“ Fourier slice photography ” , "ACM SIGGRAPH 2005 Pap.-SIGGRAPH '05, p. 735, 2005."), may be generated by processing in the Fourier transform domain.

  Also, the shift addition method (reference 3 “R. Ng, M. Levoy, G. Duval, M. Horowitz, and P. Hanrahan,“ Light Field Photography with a Hand-held Plenoptic Camera, ”Stanford Tech Rep. CTSR, pp. 1-11, 2005.), the sub-aperture image obtained from the light field image may be shifted according to the angle component, and the average image thereof may be obtained. Further, the omnifocal image may be generated by duplicating an image having a different depth of field by the above-described method and combining the focused portions. Furthermore, a central sub-aperture image that can be generated from the encoding target light field image may be set as the base image.

  Any method may be used to generate the sub-aperture image from the light field image. For example, Reference 4 “DG Dansereau, O. Pizarro, and SB Williams: Decoding, calibration and rectification for lenselet-based plenoptic cameras. In: Computer Vision and Pattern Recognition (CVPR), IEEE Conference on. IEEE, Jun 2013. "

  Next, the base image encoding unit 105 encodes the generated base image (step S105). Any method may be used for encoding. For example, an image encoding method such as JPEG or JPEG200, or a moving image encoding method such as AVC or HEVC may be used. The base image decoding unit 106 generates a decoded base image by decoding the bitstream obtained as a result of encoding by a method according to the encoded method (step S106). If the base image is lossless encoded in step S105, step S106 may be omitted and the generated base image may be a decoded base image.

  When the decoded base image is obtained, the light field image combining unit 107 generates a combined light field image for the encoding target light field image using the decoding base image and the encoding target light field image parameter (step S107). When encoding the light field image parameter to be encoded with distortion, a parameter obtained by decoding the result of step S103 is used. In addition, you may synthesize | combine using what kind of method.

  For example, it may be generated on the assumption that all the objects are completely diffusely reflected, or may be generated on the assumption that all the objects are focused on the microlens array plane. As an example of a specific method, a pixel value of a corresponding decoding base image may be assigned to a pixel group under each microlens, and may be created according to the decoding base image and an encoding target light field image parameter. A multi-viewpoint image called a sub-aperture image group may be generated, and the light-field image may be synthesized by converting the multi-viewpoint image. The pixel on the decoded base image corresponding to each microlens is identified, and the pixel You may combine by making the pixel group centering on a pixel group under the said micro lens. Instead of using the pixels of the decoded base image as they are, a pixel obtained by filtering the decoded base image according to the focus parameter for each microlens may be used.

  After generating the combined light field image, the light field image encoding unit 108 encodes the encoding target light field image while using the combined light field image as a predicted image (step S108). Any method may be used. For example, the difference between the encoding target light field image and the combined light field image may be encoded. It is also possible to encode with prediction or temporal prediction. The encoding process may be performed for each block obtained by dividing the entire image, or may be performed on the entire image. Furthermore, the prediction residual may be encoded as it is, or may be encoded using transformation such as DCT or quantization. As another method, encoding may be performed by regarding the combined light field image as an encoding target light field image in which an error has occurred due to a communication path or the like and generating an error correction code for correcting the error. .

  Finally, the multiplexing unit 109 outputs the bit stream of the base image output from the base image encoding unit 105, the bit stream of the encoding target light field image parameter output from the light field image parameter encoding unit 103, and the light field image code. The bit stream of the encoding target light field image output from the encoding unit 108 is multiplexed and output as the output of the image encoding device 100 (step S109).

  Next, the image decoding apparatus in this embodiment will be described. FIG. 3 is a block diagram showing the configuration of the image decoding apparatus according to this embodiment. As shown in FIG. 3, the image decoding apparatus 200 includes a bit stream input unit 201, a separation unit 202, a light field image parameter decoding unit 203, a base image decoding unit 204, a light field image synthesis unit 205, and a light field image decoding unit 206. It has.

  A bit stream input unit 201 inputs a bit stream of a light field image to be decoded. Hereinafter, the light field image to be decoded is referred to as a decoding target light field image. The separation unit 202 separates the input bit stream into a base image bit stream described later, a light field image parameter bit stream described later, and a decoding target light field image bit stream.

  The light field image parameter decoding unit 203 sets parameters characterizing the light field image from one of the separated bit streams. Although any parameter may be set, it is assumed that the light field image synthesis unit 205 described later sets parameters necessary for synthesizing the decoding target light field image. For example, the arrangement of microlenses in the microlens array, the rotation / parallel shift between the microlens array and the image sensor, the focus parameter of each microlens, the number of image elements corresponding to the microlens, the RGB array of the image sensor, and the decoding target Depth maps for light field images. Hereinafter, these parameters are collectively referred to as decoding target light field image parameters.

  The base image decoding 204 decodes a normal image captured at the same position / orientation / angle of view as the decoding target light field image from one of the separated bitstreams. Hereinafter, this image is referred to as a base image. The light field image synthesis unit 205 generates a synthesized light field image of the encoding target light field image from the decoded base image according to the light field image parameter. The light field image decoding unit 206 decodes the decoding target light field image from one of the separated bitstreams while using the combined light field image as a predicted image.

  Next, the operation of the image decoding apparatus 200 shown in FIG. 3 will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of the image decoding apparatus 200 shown in FIG. First, the bitstream input unit 201 inputs a bitstream obtained by encoding a decoding target light field image (step S201).

  When the input of the bit stream is completed, the separation unit 202 separates the input bit stream into a base image bit stream described later, a light field image parameter bit stream described later, and a decoding target light field image bit stream ( Step S202).

  When the bitstream separation ends, the base image decoding unit 204 decodes the base image for the light field image to be decoded from one of the separated bitstreams (step S203). The processing here is the same as in step S106 described above, and any method may be used as long as the bitstream can be correctly decoded in accordance with the method used at the time of encoding. Note that the base image obtained here may be output as one of the outputs of the image decoding apparatus 200.

  Next, the light field image parameter decoding unit 203 decodes the decoding target light field image parameter from one of the separated bit streams (step S204). Some or all of the decoding target light field image parameters may be separately input to the image decoding device 200 or may be estimated using the base image. Note that any decoding method may be used as long as it can be correctly decoded according to the method used at the time of encoding.

  When decoding of the decoding target light field image parameter and the base image is completed, the light field image combining unit 205 generates a combined light field image for the decoding target light field image (step S205). The processing here is the same as in step S107 described above, and any method may be used as long as it is the same as that at the time of encoding. If encoding distortion such as drift may occur or if the light field image to be decoded is encoded with an error correction code, a method different from that for encoding may be used.

  When the combined light field image is generated, the light field image decoding unit 206 decodes the decoding target light field image while using the combined light field image as a predicted image, and outputs the decoded light field image from the image decoding apparatus 200 (step S206). Any method may be used as long as the bitstream can be correctly decoded in accordance with the method used at the time of encoding.

  In the above-described embodiment, it is written as a process for encoding / decoding the entire image, but it can also be applied to only a part of the image. In this case, it may be determined whether or not to apply the process, and a flag indicating the process may be encoded / decoded, or may be designated by some other means. For example, you may use the method of expressing as one of the modes which show the method of producing | generating the estimated image for every area | region.

  In the above-described embodiment, one base image is used for one light field image to be processed. However, a plurality of base images may be used. For example, a plurality of sub-aperture images corresponding to a predetermined position in the sub-aperture image group corresponding to the light field image may be set as the base image. At this time, the depth map or the like for the processing target light field image that can be estimated from the plurality of sub-aperture images set as the base image may be estimated and used on both the encoding side and the decoding side without being encoded. Absent. In order to efficiently encode a plurality of base images, a multi-view video encoding method such as MV-HEVC may be used.

  FIG. 5 is a block diagram showing a hardware configuration when the above-described image encoding device 100 is configured by a computer and a software program. The system shown in FIG. 5 includes a CPU 50 that executes a program, a memory 51 such as a RAM that stores programs and data accessed by the CPU 50, and an encoding target light that inputs an image signal to be processed from a light field camera or the like. A field image input unit 52 (which may be a storage unit that stores image signals from a disk device or the like), a program storage device 53 that stores an image encoding program 531 that is a software program that causes the CPU 50 to execute image processing, and a CPU 50 A bit stream output unit 55 that outputs a bit stream generated by executing the image encoding program 531 loaded in the memory 51 (may be a storage unit that stores a bit stream by a disk device or the like) is connected via a bus. It has been configured. When the encoding target light field image parameter is given from the outside, the input unit is further connected to the bus.

  FIG. 6 is a block diagram showing a hardware configuration when the above-described image decoding apparatus 200 is configured by a computer and a software program. The system shown in FIG. 6 includes a CPU 60 that executes a program, a memory 51 such as a RAM that stores programs and data accessed by the CPU 60, and a bit stream that receives a bit stream encoded by the image encoding apparatus according to the present technique. An input unit 62 (which may be a storage unit that stores a bit stream by a disk device or the like), an image decoding program 631 that is a software program that causes the CPU 60 to execute an image decoding process, and a CPU 60 that has a memory 61 By executing the image decoding program 651 loaded on the decoding target light field image output unit 64 that outputs a decoding target light field image obtained by decoding the bitstream to a playback device or the like (an image by a disk device or the like). Remember signal It may also be) and in 憶部 but have become connected to each other by a bus. When the decoding target light field image parameter is given from the outside, the input unit is further connected to the bus. Further, when the output of the base image is necessary, the output unit is further connected to the bus.

  As described above, the image encoding device and the image decoding device according to the present embodiment encode / decode an image captured by a light field camera (recorded with rays separated for each angle). In addition, it enables efficient encoding and easy decoding of images taken by a light field camera, which cannot be efficiently encoded / decoded by a normal image encoding method / decoding method. is there. That is, by using a representative image generated from a light field image as a base layer and performing scalable coding for predictive coding of the light field image from the base layer, it becomes possible to easily check the captured contents and This realizes highly efficient compression coding of field images at the same time. According to this configuration, it is possible to easily confirm the imaging content of a light field image captured by a light field camera having a configuration in which a microlens array is inserted between the main lens and the projection surface, and highly efficient. Compression encoding can be realized.

  You may make it implement | achieve all or one part of the image coding apparatus 100 and the image decoding apparatus 200 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  When a light field image is encoded and decoded, the present invention can be applied to an application in which it is indispensable to easily confirm the contents of a scene included in the light field image and to realize efficient compression encoding at the same time.

  DESCRIPTION OF SYMBOLS 100 ... Image encoding apparatus, 101 ... Light field image input part, 103 ... Light field image parameter encoding part, 104 ... Base image generation part, 105 ... Base image encoding part, 106: Base image decoding unit, 107: Light field image synthesis unit, 108 ... Light field image encoding unit, 109 ... Multiplexing unit, 200 ... Image decoding device, 201 ... Bit stream input unit, 202 ... separation unit, 203 ... light field image parameter decoding unit, 204 ... base image decoding unit, 205 ... light field image synthesis unit, 206 ... light field image decoding , 50, 60... CPU, 51, 61... Memory, 52... Encoding target light field image input unit (storage unit), 5 63 ... Program storage device, 531 ... Image encoding program, 54 ... Bit stream output unit (storage unit), 62 ... Bit stream input unit (storage unit), 631 ... Image decoding Program, 64... Decoding target light field image output unit (storage unit)

Claims (8)

An image encoding device for encoding a first light field image obtained by a light field camera,
Light field image parameter setting means for setting a light field image parameter characterizing the first light field image based on the first light field image;
Base image generation means for generating a base image that is a normal image of the same scene as the first light field image using the first light field image;
Base image encoding means for encoding the base image;
A light field image generating means for generating a second light field image for the light field image to be encoded using the base image according to the light field image parameter;
An image encoding device comprising: a light field image encoding unit that encodes the first light field image to be encoded and outputs a bit stream while using the second light field image as a predicted image.   The image encoding device according to claim 1, wherein the base image generation unit generates an omnifocal image focused on the entire surface of the first light field image as the base image.   2. The image code according to claim 1, wherein the base image generation unit generates, as the base image, a sub-aperture image that is an image of a microlens located in the center of a main lens of the light field camera in the first light field image. Device. An image decoding device for decoding the first light field image from a bit stream of the first light field image obtained by a light field camera,
A light field image parameter setting means for decoding a parameter characterizing the first light field image and setting the decoded parameter as a light field image parameter;
Base image decoding means for decoding a base image that is a normal image of the same scene as the first light field image;
A light field image generating means for generating a second light field image for the light field image to be encoded using the base image according to the light field image parameter;
An image decoding apparatus comprising: a light field image decoding unit configured to decode the first light field image while using the second light field image as a predicted image.   The image decoding device according to claim 4, wherein the base image decoding unit decodes an omnifocal image focused on the entire image of the same scene as the first light field image as the base image.   5. The image decoding according to claim 4, wherein the base image decoding unit decodes, as the base image, a sub-aperture image that is an image of a microlens located in the center of a main lens of the light field camera in the first light field image. apparatus. An image encoding method performed by an image encoding device that encodes a first light field image obtained by a light field camera,
A light field image parameter setting step for setting a light field image parameter characterizing the first light field image based on the first light field image;
Using the first light field image to generate a base image that is a normal image of the same scene as the first light field image; and
A base image encoding step for encoding the base image;
A light field image generation step of generating a second light field image for the encoding light field image using the base image according to the light field image parameter;
And a light field image encoding step of encoding the first light field image to be encoded and outputting a bitstream while using the second light field image as a predicted image. An image decoding method performed by an image decoding apparatus that decodes the first light field image from a bit stream of a first light field image obtained by a light field camera,
A light field image parameter setting step of decoding a parameter characterizing the first light field image and setting the decoded parameter as a light field image parameter;
A base image decoding step of decoding a base image that is a normal image of the same scene as the first light field image;
A light field image generation step of generating a second light field image for the encoding light field image using the base image according to the light field image parameter;
And a light field image decoding step of decoding the first light field image while using the second light field image as a predicted image.

Images