JP2004007266A - Image encoder and its method, image decoder and its method, and program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently encode omnidirectional images obtained from multi-camera images. <P>SOLUTION: In an image encoding/decoding system 1, a JPEG-2000 encoding part 12 can perform the random access decoding of an optional camera image by regarding the omnidirectional images taken by a plurality of cameras and supplied from a camera video image input part 11 as title images respectively and performing title encoding. The JPEG-2000 encoding part 12 generates one image by collecting pixel columns or pixel rows of the same position in a plurality of camera images to improve the correlation of adjacent pixels and to improve a compression ratio. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image encoding apparatus and method for encoding images from a plurality of cameras by, for example, the JPEG-2000 system, an image decoding apparatus and method for decoding a compressed image, a program, and a recording medium.
[0002]
[Prior art]
As a conventional representative image compression method, there is a JPEG (Joint Photographic Experts Group) method standardized by ISO (International Standards Organization). It uses a discrete cosine transform (DCT) and is known to provide good encoded and decoded images when relatively high bits are assigned. However, if the number of coded bits is reduced to a certain degree or more, block distortion peculiar to DCT becomes remarkable, and deterioration is subjectively noticeable.
[0003]
On the other hand, in recent years, researches on a method of dividing an image into a plurality of bands by a filter called a filter bank, which is a combination of a high-pass filter and a low-pass filter, and performing encoding for each band have been actively conducted. Above all, wavelet transform coding is regarded as a promising new technology to replace DCT because it does not have the drawback that block distortion becomes remarkable at high compression unlike DCT.
[0004]
[Problems to be solved by the invention]
By the way, at present, electronic still cameras and video movies use the above-mentioned JPEG method and MPEG (Moving Picture Experts Group) method, and use DCT as a conversion method.
[0005]
Here, MPEG uses an intra-frame coded image (I picture) and an inter-frame coded image (P picture: unidirectional prediction, B picture: bidirectional prediction) for the purpose of improving compression efficiency. Since there is no I-picture, decoding of P-pictures and B-pictures cannot be performed. For example, there is a problem that a delay occurs when encoding an omnidirectional image by a multi-camera image, which is recently increasing in an environment such as real-life image communication and a game. Occurs. In addition, there is a restriction that a large number of frame memories for motion compensation must be provided.
[0006]
For example, DV for consumer camcorders, Motion-JPEG or DV for surveillance codecs, Motion-JPEG or DV for non-linear editing machines for professional applications, Motion-JPEG for video codecs mounted on digital cameras, etc. Still image based codecs are widely used. Reasons for this include, in addition to the small delay, the fact that the editing process is easily performed later because the image is based on a still image, and the fact that each image is independent so that it can be easily used.
[0007]
However, products based on the wavelet transform are expected to appear in the market in the future, and various research institutions are actively studying ways to improve the efficiency of the coding method. In fact, the JPEG-2000 system (working on ISO / IEC / JTC1 SC29 / WG1, which is the same organization as JPEG), which is expected to be the next generation international standard for still images, which can be said to be the successor to the JPEG system, was released in January 2001 A standardization recommendation was issued. In JPEG-2000, a wavelet transform is adopted as a conversion method which is the basis of image compression, instead of the existing JPEG DCT. In addition, JPEG-2000 is notable for achieving higher compression than JPEG, and has a great feature that JPEG-2000 has abundant functions such as progressive function, error tolerance, reversible / irreversible compression / expansion, etc. which are not available in JPEG. .
[0008]
As another feature, the Motion-JPEG2000 standard for moving images has been standardized. In this method, each image constituting a moving image is encoded as a sequence of JPEG-2000 images.
[0009]
The present invention has been proposed in view of such a conventional situation. A plurality of cameras are proposed using a JPEG-2000 or Motion-JPEG2000 system which is a still image-based coding system using a wavelet transform. Encoding Device and Method for Efficiently Encoding a Plurality of Camera Images Captured by Camera, Image Decoding Device and Method for Decoding the Compressed Image, and Image Encoding or Image Decoding Process to Computer An object of the present invention is to provide a program to be executed and a computer-readable recording medium on which the program is recorded.
[0010]
[Means for Solving the Problems]
In order to achieve the above-described object, an image encoding apparatus and method according to the present invention generates an entire image including a plurality of camera images captured by a plurality of adjacent cameras as respective components, and generates the entire image. The entire image is encoded using the camera image as a unit of encoding.
[0011]
Further, in order to achieve the above-described object, the image decoding device and the method according to the present invention provide an overall image having a plurality of camera images captured by a plurality of adjacent cameras as the respective components, and the above-described respective components. When decoding an encoded code stream generated by encoding a camera image as a unit, the encoded code stream is decoded, and the decoded camera images are displayed adjacent to each other.
[0012]
In such an image encoding apparatus and method, and an image decoding apparatus and method, a plurality of camera images are each regarded as a component of one image, and the camera image serving as the component is encoded and encoded as a unit of encoding. A plurality of subsequent camera images are displayed so as to have an adjacent original arrangement.
[0013]
In addition, in order to achieve the above-described object, an image encoding device and an image encoding method according to an aspect of the present invention include a method for combining pixels in a plurality of camera images captured by a plurality of adjacent cameras with camera images arranged adjacent to each other. Are arranged symmetrically, and the entire image generated by arranging the camera images after the arrangement in the horizontal and vertical directions is encoded.
[0014]
Further, in order to achieve the above-described object, the image decoding device and the image decoding method according to the present invention are configured such that pixels in a plurality of camera images taken by a plurality of adjacent cameras are used for connecting camera images arranged adjacent to each other. Arranged so as to be symmetrical, when decoding the encoded code stream generated by encoding the entire image in which the arranged camera images are arranged in the horizontal and vertical directions, decoding the encoded code stream, When the decoded image is divided in the horizontal and vertical directions to generate the plurality of camera images, and the pixels of the plurality of camera images are arranged so that the images arranged adjacent to each other are symmetric. To the original pixel array.
[0015]
In such an image encoding apparatus and method, and an image decoding apparatus and method, a plurality of camera images are inverted in a horizontal and vertical direction so as to be symmetrical by being inverted so that there is no large difference in pixel values at adjacent boundaries. , As one entire image. At the time of decoding, the decoded image is divided in the horizontal and vertical directions to generate the original plurality of camera images, and the pixels of the plurality of camera images are arranged so that the adjacently arranged images are symmetric. Are arranged, the original pixel arrangement is restored.
[0016]
Further, in order to achieve the above-described object, an image encoding device and method according to the present invention collectively arrange pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras. The whole image generated by the above is encoded.
[0017]
Further, in order to achieve the above-described object, an image decoding apparatus and method according to the present invention provide an image decoding apparatus and method in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged. When decoding an encoded code stream generated by encoding an image, decoding the encoded code stream and distributing the plurality of camera images by distributing pixel columns or pixel rows of the decoded image to the same position. Generate.
[0018]
In such an image encoding device and method, and an image decoding device and method, pixel columns or pixel rows at the same position of a plurality of camera images are combined and encoded as one entire image. At the time of decoding, a plurality of original camera images are generated by distributing pixel columns or pixel rows of the decoded image to the same position.
[0019]
In addition, in order to achieve the above-described object, an image encoding device and an image encoding method according to the present invention include a method of converting a first camera image captured by one or more first cameras among a plurality of adjacent cameras into a frame. Decoding the first camera image encoded by the inner encoding, and decoding the first camera image and the second camera image captured by a second camera adjacent to the first camera; Is encoded.
[0020]
Here, the image encoding apparatus and method include a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera. Is encoded.
[0021]
Further, in order to achieve the above-described object, an image decoding device and method according to the present invention include an image decoding apparatus that includes a first camera image captured by one or more first cameras among a plurality of adjacent cameras. A first encoded code stream generated by encoding, and an encoded first camera image are decoded, and the decoded first camera image and a second camera adjacent to the first camera are decoded. When decoding the first difference image with the second camera image captured by the second encoding code stream generated by encoding the first difference image, the first encoding code stream is decoded by decoding the first encoding code stream. Generating the first camera image, decoding the second encoded code stream to generate the first difference image, synthesizing the first difference image with the first camera image, and generating the first camera image. 2 camera images It is formed.
[0022]
Here, the image decoding apparatus and the image decoding method include a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera image. Decoding the encoded third encoded code stream to generate the second differential image, and combining the second differential image and the second camera image to generate the third camera image I do.
[0023]
In such an image encoding apparatus and method, and an image decoding apparatus and method, a camera image to be intra-frame encoded and a camera image to be encoded as a difference image are divided into a difference between adjacent camera images having similar image contents. And encode it. At the time of decoding, a decoded difference image and a decoded adjacent camera image are combined to generate an original camera image.
[0024]
Further, a program according to the present invention causes a computer to execute the above-described image encoding process or image decoding process, and a recording medium according to the present invention is a computer-readable medium on which such a program is recorded. is there.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In this embodiment, the present invention is applied to an image encoding apparatus and method for encoding images from a plurality of cameras in a JPEG-2000 system, and an image decoding apparatus and method for decoding the compressed image. It was done.
[0026]
(1) First embodiment
In the present embodiment, an omnidirectional image from a plurality of cameras is applied by applying a tile encoding function in the JPEG-2000 standard in which one image is divided into a plurality of rectangular regions and encoded. Tile encoding is performed by regarding each as a tile image. Here, the omnidirectional images obtained by a plurality of cameras are roughly classified into, as shown in FIG. 1A, images obtained by arranging cameras in a ring shape and capturing images from the center of the ring to the outside. As shown in FIG. 1 (B), there are two types, one obtained by arranging a camera in an annular shape and capturing an image toward the center of the annular shape. Then, as shown in FIG. 2, for example, the camera images of the first to sixteenth cameras obtained in this way are regarded as tile images, and tile encoding is performed.
[0027]
Hereinafter, before describing the image encoding / decoding system according to the present embodiment, first, wavelet transform and tile encoding in the JPEG-2000 standard will be described.
[0028]
In this wavelet transform, the low-frequency component is usually repeatedly transformed as shown in FIG. 3, because most of the energy of the image is concentrated in the low-frequency component. This means that the sub-bands are formed as shown in FIG. 4A as the division level is advanced from the division level = 1 shown in FIG. 4A to the division level = 3 shown in FIG. It can be understood from going.
[0029]
Here, the number of levels of the wavelet transform in FIG. 3 is two, and as a result, a total of seven subbands are formed. That is, the horizontal size X_SIZE and the vertical size Y_SIZE are each halved by the first filtering process, and four subbands LL-1, LH-1, HL-1, and HH-1 are generated. Is done. Then, LL-1 is further divided by the second filtering process, and four subbands of LL-2, LH-2, HL-2, and HH-2 are generated. In FIG. 3, L and H represent low and high frequencies, respectively, and the numbers after L and H represent division levels. That is, for example, LH-1 represents a subband at a division level = 1 in which the horizontal direction is low and the vertical direction is high.
[0030]
Next, tile encoding in the JPEG-2000 standard will be described. FIG. 5 shows parameters at the time of tile encoding defined by the JPEG-2000 standard. As shown in FIG. 5, one image is divided into, for example, 20 tiles whose tile indexes are T0 to T19. The horizontal size of each tile is given by XTsiz, and the vertical size is given by YTsiz. In the present embodiment, (XTsiz, YTsiz) is the horizontal size and the vertical size of the camera image. In the JPEG-2000 standard, the position of an image or a tile is expressed using a coordinate axis called a reference grid, thereby easily determining the sampling position of the image when performing resolution conversion. Can be. Here, since each tile needs to be independently encoded and decoded, it is not possible to refer to pixels beyond the tile boundary. The position of the tile is specified by a relative position between the origin of the reference grid and the upper left of the tile, and this relative position is defined as XTOsiz and YTOsiz in FIG.
[0031]
In the tile encoding of the JPEG-2000 standard, when an image in a tile is subjected to wavelet transform, when a filter protrudes to an adjacent tile area, pixels of the adjacent tile are not used, and a code as shown in FIG. Wavelet transform is performed by symmetrically expanding the pixels in the tile to be converted. In the example of FIG. 6, the pixels b and c are symmetrically extended in the horizontal direction and the pixels e and f are symmetrically extended in the horizontal direction with respect to the outside of the tile to be encoded. In the JPEG-2000 standard, there are a reversible 5 × 3 wavelet transform filter and an irreversible 9 × 7 wavelet transform filter, and the number of pixels to be symmetrically extended from a tile boundary is defined for each wavelet transform filter. Therefore, this embodiment may be followed.
[0032]
Next, FIG. 7 shows a schematic configuration of an image encoding / decoding system according to the first embodiment. As shown in FIG. 7, the image encoding / decoding system 1 includes an image encoding device 10 having a camera video input unit 11, a JPEG-2000 encoding unit 12, and a code stream recording unit 13, and a JPEG-2000 decoding unit. It comprises an image decoding device 20 having a unit 21 and an image display unit 22. The image encoding device 10 and the image decoding device 20 are connected via a network N, for example, the Internet.
[0033]
In the image encoding device 10, the camera image input unit 11 obtains omnidirectional images from a plurality of cameras and supplies the camera image information D <b> 10 to the JPEG-2000 encoding unit 12.
[0034]
The JPEG-2000 encoding unit 12 regards the camera image information D10 supplied from the camera image input unit 11 as each tile image as described above, and encodes it according to the JPEG-2000 system. The JPEG-2000 encoding unit 12 transmits the obtained encoded code stream D11 to the image decoding device 20 via the network N, or supplies the encoded code stream D11 to the code stream recording unit 13 to record the code stream on a recording medium. When there are a plurality of camera images, the encoded code stream D11 is also present by the number of cameras.
[0035]
The code stream recording unit 13 records the encoded code stream D11 supplied from the JPEG-2000 encoding unit 12 on a recording medium (not shown) such as a hard disk or a memory card.
[0036]
Note that a total of 16 camera images shown in FIG. 2 are images per time. Therefore, for example, in the case of a moving image of 30 frames per second like a normal television picture, the JPEG-2000 encoding unit 12 encodes the image of FIG. 2 at a rate of 30 frames per second as shown in FIG. I do.
[0037]
On the other hand, in the image decoding device 20, the JPEG-2000 decoding unit 21 converts the encoded code stream D11 transmitted from the image encoding device 10 or supplied from the image encoding device 10 via a recording medium into the JPEG-2000 format. Decrypt according to Here, since the presence or absence of tile encoding and parameters describing the tile size are defined in the encoded code stream D11, the JPEG-2000 decoding unit 21 faithfully decodes them. Note that tile encoding is completely independent without being affected by adjacent tiles, and thus random access decoding is possible. The JPEG-2000 decoding unit 21 supplies the obtained decoded image D20 to the image display unit 22.
[0038]
The image display unit 22 displays the decoded image D20 supplied from the JPEG-2000 decoding unit 21. Here, since the decoded images D20 exist for a plurality of cameras, the image display unit 22 displays them in the arrangement shown in FIG.
[0039]
As described above, in the first embodiment, random access decoding becomes possible by performing tile encoding by regarding each of a plurality of camera images as a tile image of one image. Since the stream can be extracted and decoded, the operability is greatly improved. For example, in an Internet environment, it is possible to decode only a camera image at a viewpoint position that a human is actually watching on the client side. This eliminates the need to decode all camera images, which is efficient.
[0040]
(2) Second embodiment
In the first embodiment, the image decoding device 20 may not be fast enough. For example, when the number of cameras is large, decoding of all of them in real time requires a large-scale LSI even if hardware is used. Therefore, there is a possibility that real-time decoding cannot be performed with an inexpensive codec LSI in terms of performance. Therefore, in the second embodiment described below, the scalability of the JPEG-2000 coded code stream is actively used. Here, the scalability means that the scale can be freely changed, that is, decoding can be performed by freely changing the resolution and image quality using one encoded code stream.
[0041]
The features of the JPEG-2000 coded code stream and specific means for achieving scalability will be described. In the JPEG-2000 system, an image is encoded using wavelet transform as described above. At this time, since the resolution of the low frequency component is hierarchically divided as shown in FIG. 3, the original encoded code stream has a multi-resolution structure.
[0042]
In the JPEG-2000 system, bit plane coding is adopted, and the data is expanded from MSB (Most Significant Bit) to LSB (Least Significant Bit) for coding. From these, in the course of encoding, as shown in FIG. 9, an encoded code stream layered from the MSB to the LSB direction can be generated.
[0043]
In the JPEG-2000 system, an encoded code stream is generated in a unit called a packet. Here, by definition, one packet is assigned to the same resolution level. In the example of FIG. 9, since the wavelet transform is performed up to the level 3, there are four packets per one layer.
[0044]
Taking advantage of these features, for example, when the performance of the image decoding device 20 is not sufficient, it is effective to decode and display only the low-frequency components (Packet-0, Packet-4, and Packet-8) in FIG. It is. In this case, since the horizontal and vertical sizes are 1/8 the size of the original image, the processing load is greatly reduced as compared with the case of decoding an encoded code stream having the size of the original image. It is possible to reduce the number of decoded images in the number of cameras in real time.
[0045]
FIG. 10 shows an example realized with actual images. As shown in FIG. 10, the resolution from the lowest band (level 0) to the highest band (level 3) is shown. Therefore, it is possible to decode the encoded code stream of the packet to a possible level according to the performance of the image decoding device 20.
[0046]
It is also effective to decode only the encoded code stream of the upper layer packet. In this case, an image having a deteriorated image quality is output at the same resolution as the original image of the camera image. FIG. 11 shows an example realized with actual images. As shown in FIG. 11, a decoded image from the lowest band (layer 0) to the highest band (layer 2) is shown. Therefore, it is possible to decode an encoded code stream of a packet up to a possible layer according to the performance of the image decoding device 20.
[0047]
As described above, in the second embodiment, the number of cameras is large by decoding the coded code stream to a possible level or to a possible layer according to the performance of the image decoding device 20. Even in this case, decoding can be performed in real time.
[0048]
(3) Third embodiment
In the present embodiment, for example, as shown in FIG. 12, a total of nine camera images of the first to ninth cameras are joined in the horizontal and vertical directions to form one image. At this time, instead of simply joining a plurality of camera images, the camera images are inverted and arranged so as to be symmetrical in advance so that there is no large difference in pixel values at adjacent boundaries, and then joined. As a result, when the wavelet transform is performed, a high-frequency component is hardly generated in an adjacent portion where an edge or the like is likely to appear, so that the overall coding efficiency is improved, and the compression rate is improved. Since the image encoding / decoding system 1 shown in FIG. 7 can be used as an actual configuration, description will be made with reference to FIG. 7 as necessary.
[0049]
The camera image input unit 11 shown in FIG. 7 obtains camera images from a plurality of cameras and supplies the camera image information D10 to the JPEG-2000 encoding unit 12.
[0050]
The JPEG-2000 encoding unit 12 joins the camera image information D10 supplied from the camera image input unit 11 in the horizontal and vertical directions as described above to form one image, and encodes this image according to the JPEG-2000 system. I do. The JPEG-2000 encoding unit 12 transmits the obtained encoded code stream D11 to the image decoding device 20 via the network N, or supplies the encoded code stream D11 to the code stream recording unit 13 to record the code stream on a recording medium.
[0051]
Note that the nine camera images in FIG. 12 are images per one time. For example, in the case of a moving image of 30 frames per second like a normal television image, the JPEG-2000 encoding unit 12 12 are encoded at a rate of 30 frames per second.
[0052]
The JPEG-2000 decoding unit 21 returns the encoded code stream D11 transmitted from the image encoding device 10 or supplied from the image encoding device 10 via the recording medium to an image before the symmetric conversion, and converts this to a JPEG-coded image. Decoding is performed according to the 2000 system. The JPEG-2000 decoding unit 21 supplies the obtained decoded image D20 to the image display unit 22.
[0053]
The image display unit 22 displays the decoded image D20 supplied from the JPEG-2000 decoding unit 21. Here, since the decoded images D20 exist for a plurality of cameras, the image display unit 22 displays them in the arrangement shown in FIG.
[0054]
As described above, in the third embodiment, a plurality of camera images are inverted so that there is no large difference in pixel values at adjacent boundaries, and are joined in the horizontal and vertical directions so as to be symmetrical to form one image. This makes it difficult to generate high-frequency components in adjacent portions where edges and the like are likely to appear when performing the wavelet transform, thereby improving the overall coding efficiency.
[0055]
(4) Fourth embodiment
In the present embodiment, for example, as shown in FIG. 13, a single image is generated by combining pixel rows at the same position of a plurality of camera images. In the example of FIG. 13, four columns are grouped in order from the left end for four camera images. As a result, the finally generated image has the same vertical size as the camera image and has a resolution four times the horizontal size. Since the image encoding / decoding system 1 shown in FIG. 7 can be used as an actual configuration, description will be made with reference to FIG. 7 as necessary.
[0056]
The camera image input unit 11 shown in FIG. 7 obtains camera images from a plurality of cameras and supplies the camera image information D10 to the JPEG-2000 encoding unit 12.
[0057]
The JPEG-2000 encoding unit 12 combines the pixel rows at the same position into one image for the camera image information D10 supplied from the camera image input unit 11 as described above, and encodes this image according to the JPEG-2000 system. Become The JPEG-2000 encoding unit 12 transmits the obtained encoded code stream D11 to the image decoding device 20 via the network N, or supplies the encoded code stream D11 to the code stream recording unit 13 to record the code stream on a recording medium.
[0058]
Here, since the characteristics of the adjacent camera images, particularly, the contents (pixel values) of the images at the same position are similar, the correlation between the adjacent pixels of the newly generated image becomes higher, and an improvement in the compression ratio can be expected.
[0059]
The JPEG-2000 decoding unit 21 converts a pixel sequence at the same position into a plurality of camera images of an encoded code stream D11 transmitted from the image encoding device 10 or supplied from the image encoding device 10 via a recording medium. A plurality of camera images are generated by distributing the image into pixel columns. Then, the JPEG-2000 decoding unit 21 decodes the plurality of camera images according to the JPEG-2000 method. The JPEG-2000 decoding unit 21 supplies the obtained decoded image D20 to the image display unit 22.
[0060]
The image display unit 22 displays the decoded image D20 supplied from the JPEG-2000 decoding unit 21. Here, since the decoded images D20 exist for a plurality of cameras, the image display unit 22 displays them in the arrangement shown in FIG.
[0061]
Note that, instead of grouping pixel columns at the same position in a plurality of camera images, for example, as shown in FIG. 14, pixel rows at the same position in a plurality of camera images can be grouped. In the example of FIG. 14, for four camera images, pixel rows at the same position of a plurality of camera images are extracted line by line from top to bottom of the images, and these are arranged in order to generate one image. As a result, the finally generated image has the same horizontal size as the camera image and a vertical size of four times the resolution.
[0062]
The JPEG-2000 encoding unit 12 generates a single image by combining pixel rows at the same position of a plurality of camera images in this way, and encodes the generated image according to the JPEG-2000 system.
[0063]
The JPEG-2000 decoding unit 21 distributes pixel rows at the same position to pixel columns of a plurality of camera images, and generates a plurality of camera images. Then, the JPEG-2000 decoding unit 21 decodes the plurality of camera images according to the JPEG-2000 method.
[0064]
The image display unit 22 displays the decoded image D20 supplied from the JPEG-2000 decoding unit 21. Here, since the decoded images D20 exist for a plurality of cameras, the image display unit 22 displays them in the arrangement shown in FIG.
[0065]
As described above, in the fourth embodiment, it is possible to increase the correlation between adjacent pixels and improve the compression ratio by generating one image by combining pixel columns or pixel rows at the same position of a plurality of camera images. It becomes possible.
[0066]
In the description of the first, third, and fourth embodiments, the JPEG-2000 encoding unit 12 encodes each image as an I picture (intra-frame encoded image) as shown in FIG. It is common to make
[0067]
(5) Fifth embodiment
In the first, third, and fourth embodiments described above, all images are encoded as I-pictures (intra-frame encoded images). However, in the present embodiment, the images to be encoded as I-pictures are The image is encoded separately from the image to be encoded as the difference image. Hereinafter, an image encoded as a difference image is referred to as an S picture. In the MPEG system, motion prediction is performed on a macroblock basis, and coding is performed on prediction errors. However, in the present embodiment, for simplification, coding is performed on a simple difference image that does not perform motion prediction. It shall be.
[0068]
First, FIG. 16 shows a conceptual diagram of an encoding method according to the present embodiment. In FIG. 15 described above, all are encoded as I pictures, whereas in FIG. 16, only the first camera is encoded as I pictures, and the other camera images are encoded as S pictures. For example, a difference between the input image of the second camera and the decoded image of the first camera is obtained, and the difference image is encoded. Similarly, the difference between the input image of the third camera and the decoded image of the second camera is obtained, and the difference image is encoded.
[0069]
Next, FIG. 17 shows a schematic configuration of an image encoding / decoding system according to the present embodiment. As shown in FIG. 17, the image encoding / decoding system 2 includes an image encoding device 30 and an image decoding device 40, and the image encoding device 30 and the image decoding device 40 are connected to each other by a network such as the Internet. N. Here, the image encoding device 30 includes a camera video input unit 31, a JPEG-2000 encoding unit 32, a JPEG-2000 decoding unit 33, a subtractor 34, a difference image encoding unit 35, and a code stream recording unit. A part 36. The image decoding device 40 includes a JPEG-2000 decoding unit 41, a difference image decoding unit 42, an adder 43, and an image display unit 44.
[0070]
In the image encoding device 30, the camera image input unit 31 obtains omnidirectional images from a plurality of cameras, and supplies the camera image information D30 to the JPEG-2000 encoding unit 32 and the subtractor 34.
[0071]
The JPEG-2000 encoding unit 32 encodes the camera image information D30 supplied from the camera video input unit 31 according to the JPEG-2000 system. The JPEG-2000 encoding unit 32 supplies the obtained encoded code stream D31 to the JPEG-2000 decoding unit 33. Further, the JPEG-2000 encoding unit 32 transmits the encoded code stream D32 encoded as an I picture to the image decoding device 40 via the network N, or supplies the encoded code stream D32 to the code stream recording unit 36.
[0072]
The JPEG-2000 decoding unit 33 decodes the encoded code stream D31 supplied from the JPEG-2000 encoding unit 32 according to the JPEG-2000 method, and generates decoded image information D33. When the camera image information D30 to be encoded as an S picture is supplied from the camera image input unit 31, the subtractor 34 subtracts the decoded image information D33 of the adjacent camera from the camera image information D30, and obtains a difference image obtained. The information D34 is supplied to the difference image encoding unit 35.
[0073]
The difference image encoding unit 35 encodes the difference image information D34 supplied from the subtractor 34 using, for example, a Live-Zempel code or an arithmetic code often used for file compression or the like. Here, the difference image information D34 is not encoded according to the JPEG-2000 method because the difference image is a kind of noise image and the correlation inside the image is very small. It is not suitable. The difference image coding unit 35 transmits the obtained coded code stream D35 to the image decoding device 40 via the network N, or supplies the coded code stream D35 to the code stream recording unit 36 to record it on a recording medium.
[0074]
On the other hand, in the image decoding device 40, the JPEG-2000 decoding unit 41 encodes an I-picture and transmits the I-picture from the image encoding device 30, or the encoded code supplied from the image encoding device 30 via a recording medium. The stream D32 is decoded according to the JPEG-2000 system. The JPEG-2000 decoding unit 41 supplies the obtained decoded image information D40 to the adder 43 and the image display unit 44.
[0075]
The difference image decoding unit 42 decodes and obtains an encoded code stream D35 encoded as an S picture and transmitted from the image encoding device 30, or supplied from the image encoding device 30 via a recording medium. The difference image information D41 is supplied to the adder 43. When the difference image information D41 is supplied from the difference image decoding unit 42, the adder 43 adds the difference image information D41 and the decoded image information D40 or the decoded image information of the adjacent camera, and obtains the obtained decoded image information. D42 is supplied to the image display unit 44.
[0076]
The image display unit 44 displays the decoded image information D40 supplied from the JPEG-2000 decoding unit 41 and the decoded image information D42 supplied from the adder 43. Here, since the decoded images exist for a plurality of cameras, the image display unit 44 displays the decoded images in the omnidirectional image arrangement shown in FIG.
[0077]
As described above, in the fifth embodiment, the image is divided into an image to be encoded as an I picture and an image to be encoded as an S picture, and encoding is performed by taking a difference between adjacent camera images having similar image contents. As a result, the compression ratio can be improved as compared with the case where only I pictures are used. On the other hand, at least a memory for holding the decoded image for reference is required, and since the encoding of the camera images in order causes a delay corresponding to the number of cameras, depending on the requirements of the system, It is preferable to use S pictures properly.
[0078]
In the example of FIG. 16, the position of the I picture is always fixed to the first camera, thereby eliminating the need for switching and simplifying the hardware control. It is not limited to fixing to one camera. For example, as shown in FIG. 18, a plurality of camera images may be encoded with an I picture, and a camera image adjacent to the I picture may be encoded with an S picture. Thus, the delay time can be reduced as compared with the case of FIG. It is also possible to switch to encode an important camera image with an I picture. For example, in the Internet environment, when transmitting a camera image at the center position of a viewpoint actually viewed by a human on the client side from the server, the camera image at the center position of the viewpoint is encoded as an I picture. Switching is preferred.
[0079]
(6) Other
The present invention is not limited to only the above-described embodiment, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.
[0080]
For example, in the above-described embodiment, an omnidirectional image is obtained by a plurality of cameras arranged in a ring, but the present invention is not limited to this. For example, a plurality of cameras arranged in an arc shape An image of the azimuth may be obtained.
[0081]
Further, in the above-described embodiment, the hardware configuration has been described. However, the present invention is not limited to this. Any processing may be realized by causing a CPU (Central Processing Unit) to execute a computer program. It is possible. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium.
[0082]
【The invention's effect】
As described in detail above, the image encoding device and method according to the present invention generate an entire image including a plurality of camera images captured by a plurality of adjacent cameras as respective components, and serve as the respective components. The entire image is encoded using the camera image as a unit of encoding.
[0083]
Further, the image decoding apparatus and method according to the present invention encodes an entire image having a plurality of camera images captured by a plurality of adjacent cameras as each component in units of the above-described component image. When decoding the generated encoded code stream, the encoded code stream is decoded, and the decoded plurality of camera images are displayed adjacent to each other.
[0084]
In such an image encoding device and method, and an image decoding device and method, each of a plurality of camera images is regarded as a component of one image, and a camera image serving as each of the components is encoded as a unit of encoding. A plurality of camera images after decoding are displayed so as to have an adjacent original arrangement. As a result, random access decoding becomes possible, and an arbitrary camera image can be extracted and decoded, so that operability is greatly improved.
[0085]
Further, the image encoding apparatus and method according to the present invention, the pixels in a plurality of camera images captured by a plurality of adjacent cameras, arranged so that adjacently arranged camera images are symmetrical, An entire image generated by arranging the arranged camera images in the horizontal and vertical directions is encoded.
[0086]
Further, the image decoding apparatus and method according to the present invention arranges pixels in a plurality of camera images captured by a plurality of adjacent cameras so that adjacently arranged camera images are symmetrical with each other. When decoding an encoded code stream generated by encoding an entire image in which the arranged camera images are arranged in the horizontal and vertical directions, the encoded code stream is decoded, and the decoded image is output in the horizontal and vertical directions. The plurality of camera images are divided in the vertical direction to generate the plurality of camera images, and when the pixels of the plurality of camera images are arranged so that adjacent images are symmetric, the original pixel arrangement is restored.
[0087]
In such an image encoding apparatus and method, and an image decoding apparatus and method, a plurality of camera images are inverted in a horizontal and vertical direction so as to be symmetrical by being inverted so that there is no large difference in pixel values at adjacent boundaries. , As one entire image. At the time of decoding, the decoded image is divided in the horizontal and vertical directions to generate the original plurality of camera images, and the pixels of the plurality of camera images are arranged so that the adjacently arranged images are symmetric. Are arranged, the original pixel arrangement is restored. This makes it difficult for high-frequency components to be generated in adjacent portions where edges and the like are likely to appear, thereby improving the overall coding efficiency.
[0088]
Further, the image encoding device and method according to the present invention encode an entire image generated by arranging pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras collectively. .
[0089]
In addition, the image decoding apparatus and method according to the present invention include an encoding apparatus that encodes and generates an entire image in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged. When decoding the code stream, the plurality of camera images are generated by decoding the encoded code stream and distributing pixel columns or pixel rows of the decoded image to the same position.
[0090]
In such an image encoding device and method, and an image decoding device and method, pixel columns or pixel rows at the same position of a plurality of camera images are combined and encoded as one entire image. At the time of decoding, a plurality of original camera images are generated by distributing pixel columns or pixel rows of the decoded image to the same position. This makes it possible to increase the correlation between adjacent pixels and improve the compression ratio.
[0091]
Further, the image encoding apparatus and method according to the present invention is characterized in that the first camera image captured by one or two or more first cameras among a plurality of adjacent cameras is intra-frame encoded, and the encoded Decoding a first camera image and encoding a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera image; I do.
[0092]
Here, the image encoding apparatus and method include a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera. Is encoded.
[0093]
In addition, the image decoding apparatus and method according to the present invention include a first code generated by intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras. An encoded code stream and an encoded first camera image, and the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera When decoding a second encoded code stream generated by encoding a first differential image with the first encoded image, the first encoded image is generated by decoding the first encoded code stream. The second encoded code stream is decoded to generate the first differential image, and the first differential image and the first camera image are combined to generate the second camera image.
[0094]
Here, the image decoding apparatus and the image decoding method include a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera image. Decoding the encoded third encoded code stream to generate the second differential image, and combining the second differential image and the second camera image to generate the third camera image I do.
[0095]
In such an image encoding apparatus and method, and an image decoding apparatus and method, a camera image to be intra-frame encoded and a camera image to be encoded as a difference image are divided into a difference between adjacent camera images having similar image contents. And encode it. At the time of decoding, a decoded difference image and a decoded adjacent camera image are combined to generate an original camera image. This makes it possible to improve the compression ratio as compared with the case where all camera images are intra-frame encoded.
[0096]
Further, a program according to the present invention causes a computer to execute the above-described image encoding process or image decoding process, and a recording medium according to the present invention is a computer-readable medium on which such a program is recorded. is there.
[0097]
According to such a program and a recording medium, the above-described image encoding process or image decoding process can be realized by software.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining an omnidirectional image according to a first embodiment. FIG. 1A shows an example in which a camera is arranged on a concentric circle and an image is taken in an external direction from the center of the concentric circle. FIG. 2B shows an example in which a camera is arranged on a concentric circle and an image is taken toward the center of the concentric circle.
FIG. 2 is a diagram illustrating an example in which a plurality of camera images are regarded as tile images in the first embodiment.
FIG. 3 is a diagram illustrating subbands when wavelet transform is performed up to a second level.
FIGS. 4A and 4B are diagrams illustrating subbands when an actual image is subjected to wavelet transform. FIG. 4A shows an example of division into a first level, and FIG. An example of division is shown.
FIG. 5 is a diagram illustrating parameters at the time of tile encoding according to the first embodiment.
FIG. 6 is a diagram illustrating a range covered by a wavelet transform in the tile encoding.
FIG. 7 is a diagram illustrating a schematic configuration of an image encoding / decoding system according to the first embodiment.
FIG. 8 is a diagram illustrating encoding of a tile image group in the time direction.
FIG. 9 is a diagram illustrating a layer structure and packet arrangement when an encoded code stream is divided into a plurality of layers in the second embodiment.
FIG. 10 is a diagram illustrating resolution progressive in an actual image.
FIG. 11 is a diagram illustrating image quality progressive in an actual image.
FIG. 12 is a diagram illustrating an example in which a plurality of camera images are arranged in a symmetrical relationship in the third embodiment.
FIG. 13 is a diagram illustrating an example in which pixels in the same column of a plurality of camera images are combined into one image in the fourth embodiment.
FIG. 14 is a diagram illustrating an example in which pixels in the same row of a plurality of camera images are combined into one image in the fourth embodiment.
FIG. 15 is a diagram illustrating an example in which all camera images are encoded as I pictures.
FIG. 16 is a diagram illustrating an example in which one camera image among a plurality of camera images is encoded as an I picture and the other camera images are encoded as S pictures in the fifth embodiment.
FIG. 17 is a diagram illustrating a schematic configuration of an image encoding / decoding system according to the fifth embodiment.
FIG. 18 is a diagram illustrating an example in which, in the fifth embodiment, a plurality of camera images among a plurality of camera images are encoded as I-pictures, and other camera images are encoded as S-pictures.
[Explanation of symbols]
1, 2 image encoding / decoding system, 10 image encoding device, 11 camera video input unit, 12 JPEG-2000 encoding unit, 13 code stream recording unit, 20 image decoding device, 21 JPEG-2000 decoding unit, 22 image display Unit, 30 image encoding device, 31 camera video input unit, 32 JPEG-2000 encoding unit, 33 JPEG-2000 decoding unit, 34 subtractor, 35 difference image encoding unit, 36 code stream recording unit, 40 image decoding device , 41 JPEG-2000 decoding unit, 42 difference image decoding unit, 43 adder, 44 image display unit

Claims (56)

Means for generating an overall image having a plurality of camera images captured by a plurality of adjacent cameras as respective constituent elements,
Means for encoding the entire image using the camera image as each of the constituent elements as a unit of encoding. The image encoding device according to claim 1, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. 2. The image encoding apparatus according to claim 1, wherein said encoding means encodes the entire image in a frame. 4. The image encoding apparatus according to claim 3, wherein said encoding means performs tile encoding of the plurality of camera images as tile images in a JPEG-2000 format. Means for arranging pixels in a plurality of camera images captured by a plurality of adjacent cameras so that camera images arranged adjacent to each other are symmetric;
Means for arranging the arranged camera images in the horizontal and vertical directions to generate an entire image,
Means for encoding the entire image. The image encoding apparatus according to claim 5, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. Means for collectively arranging pixel columns or pixel rows at the same position of a plurality of camera images taken by a plurality of adjacent cameras, and generating an entire image;
Means for encoding the entire image. The image encoding apparatus according to claim 7, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. Means for intra-frame encoding a first camera image taken by one or more first cameras among a plurality of adjacent cameras;
Means for decoding the encoded first camera image;
Means for encoding a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera. Image encoding device. The apparatus further comprises means for encoding a second difference image between the decoded second camera image and a third camera image taken by a third camera adjacent to the second camera. The image encoding device according to claim 9, wherein The image encoding apparatus according to claim 9, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. The image encoding device according to claim 9, wherein the position of the first camera is always constant. 10. The image encoding apparatus according to claim 9, wherein the position of the first camera is a center position of a viewpoint. A step of generating an overall image having a plurality of camera images captured by a plurality of adjacent cameras as respective components,
Encoding the entire image using the camera image as each component as an encoding unit. A step of arranging pixels in a plurality of camera images captured by a plurality of adjacent cameras so that camera images arranged adjacent to each other are symmetrical,
A step of arranging the arranged camera images in the horizontal and vertical directions to generate an entire image,
Encoding the entire image. A step of collectively arranging pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras, and generating an entire image,
Encoding the entire image. Intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras;
Decoding the encoded first camera image;
Encoding a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera. Image encoding method. Encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera image. 18. The image encoding method according to claim 17, wherein In a program for causing a computer to execute a predetermined process,
A step of generating an overall image having a plurality of camera images captured by a plurality of adjacent cameras as respective components,
Encoding the entire image using the camera image as each of the constituent elements as an encoding unit. In a program for causing a computer to execute a predetermined process,
A step of arranging pixels in a plurality of camera images captured by a plurality of adjacent cameras so that camera images arranged adjacent to each other are symmetrical,
A step of arranging the arranged camera images in the horizontal and vertical directions to generate an entire image,
Encoding the entire image. In a program for causing a computer to execute a predetermined process,
A step of collectively arranging pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras, and generating an entire image,
Encoding the entire image. In a program for causing a computer to execute a predetermined process,
Intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras;
Decoding the encoded first camera image;
Encoding a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera. Program to do. Encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera image. The program according to claim 22, which performs the program. In a computer-readable recording medium in which a program for causing a computer to execute a predetermined process is recorded,
A step of generating an overall image having a plurality of camera images captured by a plurality of adjacent cameras as respective components,
Encoding the entire image using the camera image as each component as a unit of encoding. In a computer-readable recording medium in which a program for causing a computer to execute a predetermined process is recorded,
A step of arranging pixels in a plurality of camera images captured by a plurality of adjacent cameras so that camera images arranged adjacent to each other are symmetrical,
A step of arranging the arranged camera images in the horizontal and vertical directions to generate an entire image,
Encoding the whole image. A recording medium on which a program is recorded. In a computer-readable recording medium in which a program for causing a computer to execute a predetermined process is recorded,
A step of collectively arranging pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras, and generating an entire image,
Encoding the whole image. A recording medium on which a program is recorded. In a computer-readable recording medium in which a program for causing a computer to execute a predetermined process is recorded,
Intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras;
Decoding the encoded first camera image;
Encoding a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera. A recording medium on which a program to be recorded is recorded. The program further includes a step of encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera. 28. The recording medium according to claim 27, wherein: An image decoding apparatus that decodes an encoded code stream generated by encoding an entire image including a plurality of camera images captured by a plurality of adjacent cameras as components, and encoding the entire image in units of the camera images as the components. And
Means for decoding the encoded code stream;
Means for displaying the plurality of decoded camera images adjacent to each other. 30. The image decoding apparatus according to claim 29, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. 30. The image decoding apparatus according to claim 29, wherein the decoding unit performs tile decoding using the plurality of camera images as tile images in a JPEG-2000 format. 30. The image decoding apparatus according to claim 29, wherein said decoding means selects a camera image to be decoded according to a viewpoint position. 30. The image decoding apparatus according to claim 29, wherein said decoding means decodes only an encoded code stream of a low-frequency component of a camera image when a decoding speed is not sufficient. 30. The image decoding apparatus according to claim 29, wherein said decoding means decodes only an encoded code stream of an upper layer of a camera image when a decoding speed is not sufficient. Pixels in a plurality of camera images captured by a plurality of adjacent cameras are arranged so that the camera images arranged adjacent to each other are symmetrical, and the arranged camera images are arranged in a horizontal direction and a vertical direction. An image decoding device that decodes an encoded code stream generated by encoding the entire image that has been generated,
Means for decoding the encoded code stream;
Means for dividing the decoded image in the horizontal direction and the vertical direction to generate the plurality of camera images,
Means for returning to the original pixel array when pixels of the plurality of camera images are arranged so that adjacently arranged images are symmetrical. 36. The image decoding device according to claim 35, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. An image decoding apparatus that decodes an encoded code stream generated by encoding an entire image in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged,
Means for decoding the encoded code stream;
Means for generating the plurality of camera images by distributing pixel columns or pixel rows of the decoded image to the same position. 38. The image decoding device according to claim 37, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. A first encoded code stream generated by intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras, and the first encoded code stream A camera image is decoded, and a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera is generated by encoding. An image decoding device that decodes a second encoded code stream,
Means for decoding the first encoded code stream to generate the first camera image;
Means for decoding the second encoded codestream to generate the first differential image;
Means for synthesizing the first difference image and the first camera image to generate the second camera image. A third encoded code stream obtained by encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera is obtained. Means for decoding to generate the second difference image;
40. The image decoding apparatus according to claim 39, further comprising: means for combining the second difference image and the second camera image to generate the third camera image. 40. The image decoding apparatus according to claim 39, wherein the plurality of cameras are arranged in a ring shape, and capture an image in a center direction or an outer direction of the ring. An image decoding method for decoding an encoded code stream generated by encoding an entire image having a plurality of camera images captured by a plurality of adjacent cameras as constituent elements in units of the camera images as the constituent elements And
Decoding the encoded codestream;
Displaying the plurality of decoded camera images adjacent to each other. Pixels in a plurality of camera images captured by a plurality of adjacent cameras are arranged so that the camera images arranged adjacent to each other are symmetrical, and the arranged camera images are arranged in a horizontal direction and a vertical direction. An image decoding method for decoding an encoded code stream generated by encoding the entire image,
Decoding the encoded codestream;
Generating the plurality of camera images by dividing the decoded image in a horizontal direction and a vertical direction,
When the pixels of the plurality of camera images are arranged such that adjacently arranged images are symmetrical, the pixel arrangement is returned to the original pixel arrangement. An image decoding method for decoding an encoded code stream generated by encoding an entire image in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged,
Decoding the encoded codestream;
Generating the plurality of camera images by distributing pixel columns or pixel rows of the decoded image to the same position. A first encoded code stream generated by intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras, and the first encoded code stream A camera image is decoded, and a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera is generated by encoding. An image decoding method for decoding a second encoded code stream, comprising:
Decoding the first encoded codestream to generate the first camera image;
Decoding the second encoded codestream to generate the first difference image;
Combining the first difference image and the first camera image to generate the second camera image. A third encoded code stream obtained by encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera is obtained. Decoding to generate the second difference image;
46. The image decoding method according to claim 45, further comprising: combining the second difference image and the second camera image to generate the third camera image. An image decoding process for decoding an encoded code stream generated by encoding an entire image having a plurality of camera images captured by a plurality of adjacent cameras as constituent elements in units of the camera images as the constituent elements. Is a program that causes a computer to execute
Decoding the encoded codestream;
Displaying the plurality of decoded camera images adjacent to each other. Pixels in a plurality of camera images captured by a plurality of adjacent cameras are arranged so that the camera images arranged adjacent to each other are symmetrical, and the arranged camera images are arranged in a horizontal direction and a vertical direction. A program that causes a computer to execute an image decoding process of decoding an encoded code stream generated by encoding the entire image that has been generated,
Decoding the encoded codestream;
Generating the plurality of camera images by dividing the decoded image in a horizontal direction and a vertical direction,
When the pixels of the plurality of camera images are arranged so that adjacently arranged images are symmetrical, the image is returned to the original pixel array. The computer executes an image decoding process of decoding an encoded code stream generated by encoding an entire image in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged. A program,
Decoding the encoded codestream;
Generating the plurality of camera images by distributing pixel columns or pixel rows of the decoded image to the same position. A first encoded code stream generated by intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras, and the first encoded code stream A camera image is decoded, and a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera is generated by encoding. A program for causing a computer to execute an image decoding process of decoding a second encoded code stream,
Decoding the first encoded codestream to generate the first camera image;
Decoding the second encoded codestream to generate the first difference image;
Combining the first difference image and the first camera image to generate the second camera image. A third encoded code stream obtained by encoding a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera is obtained. Decoding to generate the second difference image;
The program according to claim 50, further comprising the step of: combining the second difference image and the second camera image to generate the third camera image. An image decoding process for decoding an encoded code stream generated by encoding an entire image having a plurality of camera images captured by a plurality of adjacent cameras as constituent elements in units of the camera images as the constituent elements. Is a computer-readable recording medium on which a program for causing a computer to execute is recorded,
Decoding the encoded codestream;
Displaying the plurality of decoded camera images adjacent to each other. Pixels in a plurality of camera images taken by a plurality of adjacent cameras are arranged so that adjacently arranged camera images are symmetrical, and the arranged camera images are arranged in the horizontal and vertical directions. A computer-readable recording medium recorded with a program for causing a computer to execute an image decoding process of decoding an encoded code stream generated by encoding the entire image,
Decoding the encoded codestream;
Generating the plurality of camera images by dividing the decoded image in a horizontal direction and a vertical direction,
A step of returning to the original pixel array when pixels of the plurality of camera images are arranged so that adjacently arranged images are symmetrical. . The computer executes an image decoding process of decoding an encoded code stream generated by encoding an entire image in which pixel columns or pixel rows at the same position of a plurality of camera images captured by a plurality of adjacent cameras are collectively arranged. A computer-readable recording medium on which the program is recorded,
Decoding the encoded codestream;
Generating the plurality of camera images by distributing pixel columns or pixel rows of the decoded image to the same position. A first encoded code stream generated by intra-frame encoding a first camera image captured by one or more first cameras among a plurality of adjacent cameras, and the first encoded code stream A camera image is decoded, and a first difference image between the decoded first camera image and a second camera image captured by a second camera adjacent to the first camera is generated by encoding. A computer-readable recording medium on which a program for causing a computer to execute an image decoding process for decoding the second encoded code stream and a computer is recorded.
Decoding the first encoded codestream to generate the first camera image;
Decoding the second encoded codestream to generate the first difference image;
A step of combining the first difference image and the first camera image to generate the second camera image. The program includes a third code that encodes a second difference image between the decoded second camera image and a third camera image captured by a third camera adjacent to the second camera. Decoding the encoded code stream to generate the second difference image;
The recording medium according to claim 55, further comprising a step of generating the third camera image by combining the second difference image and the second camera image.

Images