JP4251864B2 - Image data creating apparatus and image data reproducing apparatus for reproducing the data - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image data creation apparatus that attaches attribute information to image data when creating image data for three-dimensional display, and an image data reproduction apparatus that reproduces the data.
[0002]
[Prior art]
Conventionally, various methods for displaying a three-dimensional image have been proposed. Among them, what is generally used is a so-called “binocular type” that uses binocular parallax. That is, a left-eye image and a right-eye image having binocular parallax are prepared, and stereoscopic vision is performed by projecting them to the left and right eyes independently.
[0003]
FIG. 26 is a conceptual diagram for explaining a “time division method”, which is one of the typical two-lens methods.
[0004]
In this time division method, as shown in FIG. 26, the left eye image and the right eye image are alternately arranged every other pixel in the vertical direction, and the display of the left eye image and the display of the right eye image are switched alternately. Is displayed. The left-eye image and the right-eye image have a vertical resolution that is ½ that of normal two-dimensional display. The observer wears shutter-type glasses that open and close in synchronization with the display switching cycle. The shutter used here opens the left eye side and closes the right eye side when the left eye image is displayed, and closes the left eye side and opens the right eye side when the right eye image is displayed. By doing so, the left eye image is observed only by the left eye, and the right eye image is observed only by the right eye, and stereoscopic viewing can be performed.
[0005]
FIG. 27 is a conceptual diagram for explaining a “parallax barrier method” which is another representative method of the two-lens method.
[0006]
FIG. 27A is a diagram illustrating the principle of generating parallax. On the other hand, FIG. 27B is a diagram showing a screen displayed by the parallax barrier method.
[0007]
In FIG. 27A, an image in which the left eye image and the right eye image as shown in FIG. 27B are alternately arranged every other pixel in the horizontal direction is displayed on the image display panel 401. By placing a parallax barrier 402 having slits at intervals smaller than the interval between pixels of the same viewpoint on the front surface of the image display panel 401, the left eye image is observed only by the left eye 403, and the right eye image is observed only by the right eye 404. That is, stereoscopic viewing can be performed.
[0008]
Incidentally, as in the parallax barrier method, there is a “lenticular method” as a method for displaying an image as shown in FIG. 27B three-dimensionally. An example of a recording data format for use in this lenticular method is disclosed in Patent Document 1 described later.
[0009]
FIG. 28 is a conceptual diagram showing an example of such a lenticular recording data format. Each of the left eye image 501 shown in FIG. 28 (a) and the right eye image 502 shown in FIG. 28 (b) is thinned out in half in the horizontal direction, and one mixed image 503 shown in FIG. 28 (c). Make and record. At the time of reproduction, the mixed image 503 is rearranged to create a composite image as shown in FIG.
[0010]
[Patent Document 1]
Japanese Patent Laid-Open No. 11-41627
[0011]
[Problems to be solved by the invention]
As described above, in the conventional three-dimensional display system, recording with a fixed recording data format is performed so as to be suitable for the display method determined on the reproducing apparatus side, and the recording data should be versatile. Is not considered.
[0012]
In addition to the display method, there are various information necessary for three-dimensional display such as the image thinning method and the number of viewpoints in the so-called “multi-view method”, but when the display method is single, such information is recorded as recorded data. Not recorded. If the same display method is always used, it is not necessary to record such information. However, the versatility of recorded data is greatly impaired. For example, even when recording data for the parallax barrier method (or lenticular method), the left-eye image and the right-eye image can be recorded as separate sequences, as shown in FIG. It is also possible to record a mixed image in which the eye image and the right eye image are arranged on the left and right of the screen half, or a composite image in which the left eye image and the right eye image are arranged every other pixel in the horizontal direction as shown in FIG. Can also be recorded. Of course, if the recording format is different, the processing method for displaying this will also be different, but since it is not possible to know which format was recorded from the recorded data, when a third party obtains the data, which There is a problem that it is not known whether the display should be performed by such processing.
[0013]
Also, as shown in FIG. 29, in the synthesized image created by the conventional technique, the pixel values are usually discontinuous at the boundary portion between the synthesized left eye image and right eye image. For this reason, there is a problem that the encoding efficiency is lowered when encoding such as JPEG. For example, when a boundary between a left eye image and a right eye image exists in a DCT block in JPEG, the high frequency component of the DCT transform coefficient is increased, resulting in a decrease in encoding efficiency. Furthermore, when MPEG-4 is used as the encoding method, the encoding efficiency is improved by predicting the DCT coefficient of the encoding target block from the neighboring blocks in the same frame. Generally, the DCT of each viewpoint image is improved. Since the coefficients are discontinuous around each boundary part, even when the boundary part of each viewpoint image does not exist in the DCT block, the prediction of the DCT coefficient is not successful and the coding efficiency may be reduced.
[0014]
Further, in the conventional technique, when image data of an arbitrary viewpoint is recorded, it is not considered that an image of a desired viewpoint is easily read and reproduced.
[0015]
The present invention has been made to solve the above-described problems, and its purpose is to provide versatility to image data for three-dimensional display and to efficiently image an arbitrary viewpoint. In addition, when encoding an image obtained by combining a plurality of images adjacent to each other, it is possible to select an image data creation device that enables efficient encoding, and image data for reproducing the data. To provide a playback device.
[0016]
[Means for Solving the Problems]
To achieve the above object, the present invention includes a plurality of images corresponding to each of a plurality of viewpoints that can be stereoscopically viewed. For stereoscopic viewing In an image data creation device for creating image data, An image used when reproducing the stereoscopic image data as a 2D image, The plurality of viewpoints User from inside Select one 2D showing the selected viewpoint Generate selection information 2D Selection information generating means, For stereoscopic viewing Image data is 2D generated by the 2D selection information generating means Selection information Multiplexed Thus, the above-described problem is solved.
[0017]
In addition, the present invention includes a plurality of images corresponding to each of a plurality of viewpoints that can be stereoscopically viewed. For stereoscopic viewing A plurality of images corresponding to each of the plurality of viewpoints from the image data Image data for stereoscopic viewing based on In an image data reproducing apparatus provided with an image reproducing means for reproducing, For stereoscopic viewing Image data includes Images used for playback as 2D images The plurality of viewpoints From inside Select one 2D for information Selection information Multiplexed The image reproduction means For stereoscopic viewing From image data 2D Analyzing means for analyzing selection information; and 2D The image corresponding to the viewpoint selected by the selection information As a 2D image The above-described problem is solved by having selected image reproducing means for reproducing.
[0044]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0045]
<First Embodiment>
FIG. 1 is a block diagram showing the configuration of an image data creation apparatus according to the first embodiment of the present invention. In FIG. 1, an image data creation device 100 creates an image combining unit 101 that generates an image obtained by combining a left-eye image and a right-eye image, and whether or not to combine the left-eye image and the right-eye image (whether or not combined) ), Whether to reduce the left-eye image and the right-eye image at the time of image combination (presence / absence of reduction), and a control unit 102 that designates an image to be used for two-dimensional display (2D selection), A 3D information creation unit 103 that formats information on presence / absence, presence / absence of coupling, and 2D selection to create 3D information, and a means for accessing a recording medium and a communication line, and multiplexing and outputting image data and 3D information The unit 104 is configured.
[0046]
The operation of the image data creation device 100 configured as described above will be described.
[0047]
An image signal composed of continuous frames is input to the image data creation device for each frame.
The control unit 102 designates whether or not to reduce, whether or not to combine, and 2D selection. The presence / absence of reduction takes one of the values “with reduction” or “no reduction”, and the presence / absence of connection takes the value of “without connection” or “with connection”. In the 2D selection, either one of the left and right images may be specified, or may not be specified in particular.
[0048]
FIG. 2 is a diagram showing a combination of presence / absence of reduction and presence / absence of coupling. As shown in FIG. 2, there are four combinations in total.
[0049]
The image combining unit 101 creates image data in a format specified by the presence / absence of reduction and the presence / absence of combination. These combinations are shown in FIG. 3A shows the case of “no reduction” and “no connection”, FIG. 3B shows the case of “no reduction” and “with connection”, and FIG. FIG. 3D shows the case of “no connection”, and the case of “with reduction” and “with connection”. Here, in the case of “with reduction”, the resolution is halved by thinning out pixels or the like. The reduction ratio may be fixed or variable. If variable, the reduction ratio is recorded in 3D information.
[0050]
The 3D information creation unit 103 formats the presence / absence of reduction, the presence / absence of combination, and 2D selection to create 3D information necessary for displaying an image in three dimensions. As for the presence / absence of reduction and the presence / absence of combination, the index shown in FIG. 2 may be recorded as it is, or first information indicating presence / absence of reduction is recorded, and then second information indicating presence / absence of combination is recorded. May be. Further, the 3D information may be encoded by fixed length encoding or variable length encoding.
[0051]
The multiplexing unit 104 converts the image data and the 3D information into a predetermined format and outputs them to the outside. Although not shown in FIG. 1, when voice and music are multiplexed, those data are also multiplexed by the multiplexing unit 104.
[0052]
An output destination of the multiplexing unit 104 is connected to a recording device such as an IC memory, a magneto-optical disk, a magnetic tape, and a hard disk, and a communication device such as a LAN and a modem. Here, it is assumed that an IC memory is connected to the multiplexing unit 104. Hereinafter, a recording format in this case will be described.
[0053]
In general, when an IC memory is used as a recording medium, a file system such as FAT (File Allocation Table) is constructed on the IC memory, and data is recorded as a file. As the file format used here, an existing format may be used, or a newly defined unique format may be used.
[0054]
FIG. 5 is a diagram showing a file format for recording multiplexed data. In FIG. 5, data is recorded in a file in the order from the top to the bottom of the figure. FIG. 5A is an example when the existing format is used, and FIG. 5B is an example when the new format is used.
[0055]
When an existing format is used, 3D information is generally recorded as a part of an existing header portion using a mechanism for extending a header portion prepared in the existing format. Here, the extended header is called an extension header. For example, in JPEG, the file header indicates an application data segment, a new application data segment is defined, and 3D information is recorded. In MPEG-4, the file header indicates Visual Object Sequence or / and Video Object Layer, and 3D information is recorded as user data in these.
[0056]
If an existing format is used, a commonly used extension is used as it is. For example, in the case of a JPEG file, generally. If the extension jpg is an MPEG file, generally mpg or. When the extension of mp4 is WMV (Windows (R) Media Video), generally. The extension wmv is used. In this way, even a conventional playback device that does not have a three-dimensional image display function can recognize the file as an existing format and display it as a two-dimensional image.
[0057]
On the other hand, when using a new format, for example, 3D information is recorded at the head of the file as shown in FIG. In addition, a unique extension that can be distinguished from an existing format file is attached so that the file is a new format file. Note that the management information in FIGS. 5A and 5B is used for recording information that is not directly related to the three-dimensional image, such as the creation date and the creator.
[0058]
Here, how to store the left and right images will be described by taking as an example the case where the existing format shown in FIG. 5A is used as the file format. When the presence / absence of combination indicates “with combination”, the combined image data is recorded in the image data area of FIG. 5A. In the case of a moving image, a plurality of images obtained by combining corresponding frames of the left and right images are recorded. At this time, each frame may be encoded independently as in Motion JPEG, or the difference may be encoded using inter-frame prediction as in MPEG-4. When the presence / absence of combination indicates “no combination”, the left and right two image data may be recorded separately in the image data area of FIG. 5A, or may be recorded as separate files. . The format at this time is shown in FIG. In FIG. 6A, left and right image data are recorded in one file. In FIG. 6B, the left and right image data are recorded in separate files in the format of FIG. In the case of moving images, a plurality of pieces of frame data are recorded. Although the arrangement order of the left and right images and the storage order in the file have been described, these orders may be variable. If variable, the order information is recorded in the 3D information. In FIG. 6A, data other than image data may be recorded between two pieces of image data.
[0059]
Next, a playback device for displaying the image data created by the image data creation device 100 as a three-dimensional image will be described.
[0060]
FIG. 8 is a block diagram showing the configuration of the image data reproducing apparatus according to the first embodiment of the present invention. In FIG. 8, an image data reproduction device 200 includes a demultiplexing unit 201 that separates multiplexed data, a 3D information analysis unit 202 that analyzes 3D information, and an image conversion unit 203 that converts image data into a display format. .
[0061]
The operation of the image data reproducing apparatus 200 configured as described above will be described.
[0062]
The demultiplexing unit 201 reads multiplexed data multiplexed in a predetermined format from a recording device or a communication device, and separates it into image data and 3D information. Although not shown in FIG. 8, when voice and music are multiplexed, those data are also separated by the demultiplexer 201. Here, it is assumed that an IC memory is connected to the demultiplexer 201. As described above, an image file is recorded in the IC memory in an existing format or a new format. Since the existing format and the new format can be distinguished by the extension of the file, when the file to be reproduced is the file of the existing format shown in FIG. 5A, 3D information is read from the extension area of the file header. In the case of the new format shown in FIG. 5B, 3D information is read from the beginning of the file.
[0063]
The 3D information analysis unit 202 analyzes the 3D information, and extracts the presence / absence of reduction, the presence / absence of combination, and the setting value of 2D selection.
[0064]
The image conversion unit 203 is connected to a display device having a different display format, such as a two-dimensional display device using a normal cathode ray tube or a liquid crystal panel, a stereoscopic display device using a lenticular method, a parallax barrier method, a time division method, or the like. The The image conversion unit 203 converts the separated image data into a display format based on the presence / absence of reduction and the presence / absence of combination. When a parallax barrier display device is connected, the format shown in FIG. 3D is the easiest to handle. In this case, the left and right images need only be arranged every other pixel in the horizontal direction. When the image data is in the format shown in FIG. 3B, the left and right images are thinned out in half in the horizontal direction before being arranged every other pixel in the horizontal direction. In either case, since 3D information is added to the image data, it can be converted into a display format suitable for the display device.
[0065]
When the display is switched to the two-dimensional display during the display of the three-dimensional image, the image designated by the 2D selection (that is, one of the left eye image and the right eye image) is displayed. At this time, when the presence / absence of reduction is “no reduction”, the image is displayed as it is, and when it is “reduction”, the image is enlarged twice. However, when 2D selection does not specify an image to be used for two-dimensional display, an image to be displayed is selected by a predetermined method.
[0066]
When a 2D display device is connected, an image designated by 2D selection is displayed. The way of display is the same as the two-dimensional display in the three-dimensional display device.
[0067]
In the above embodiment, when the presence / absence of combination indicates “with combination”, the image combination unit 101 in FIG. 1 combines two images in the horizontal direction, but the combination direction is the vertical direction. There may be. The coupling result at this time is shown in FIG. FIG. 4A shows the case of “with reduction”, and FIG. 4B shows the case of “without reduction”. 3 (c) and 3 (d) are reduced to ½ in the horizontal direction, this is a format suitable for the parallax barrier method described above, and FIG. 4 (b) is reduced to ½ in the vertical direction. Therefore, it can be said that the format is suitable for the time division method described above. In any case, by adding 3D information to the image data, it can be converted into a display format suitable for the display device.
[0068]
In the above embodiment, the case where image data is recorded in a file of an existing format has been described. Next, the case of recording in a file of a new format will be described. The new format may use an existing format (JPEG, bitmap, etc.) or a completely new unique format for the file header and image data portion in FIG. 5B. Therefore, in order to distinguish between these formats, type information (referred to as an image type) is recorded in the 3D information.
[0069]
Regarding how to record image data. When the presence / absence of combination indicates “with combination”, the combined image data is recorded in the image data area of FIG. 5B. In the case of a moving image, as described above, a plurality of images obtained by combining corresponding frames of the left and right images are recorded. When the presence / absence of the combination indicates “no combination”, the left and right image data are recorded in the image data area of FIG. In the case of moving images, a plurality of pieces of frame data are recorded. FIG. 7 shows an example of the file structure and 3D information at this time. In FIG. 7A, left and right image data are recorded in one file. In FIG. 7 (b), the left and right image data are recorded in separate files in the format of FIG. 5 (b). FIG. 7B also records the same 3D information as shown in FIG.
In FIG. 7A, the two sets of file header and image data are in a format that can be recognized as a file of an existing format by itself. In other words, taking a bitmap file as an example, the left eye image and the right eye image are recorded as bitmap files, respectively, and 3D information, management information, the left eye image bitmap file, and the right eye image bitmap file are connected in order. It is a format.
[0070]
Although the arrangement order of the left and right images and the storage order in the file have been described, these orders may be variable. If variable, the order information may be recorded in 3D information, or may be recorded in an existing format file header using the above-described extension header. In addition, the case where 3D information and management information are recorded at the beginning of the file has been described. However, the new format file is not limited to this, and the storage position thereof is also after the file header or after the image data. It may be the same as the existing format shown in FIG.
[0071]
In the above embodiment, the case where the number of viewpoints is two has been described. However, the present invention can also be applied to a so-called multi-viewpoint having three or more viewpoints. Here, M imaging devices for inputting image data to the image data creation device 100 are arranged in a grid in the horizontal direction and M in the horizontal direction and N in the vertical direction, and each imaging device has a number ( (Viewpoint number) is assigned (where M and N are integers of 1 or more). FIG. 9 shows an installation example in the case where the number of viewpoints is 8 (a diagram in which the installed imaging apparatus is looked down obliquely downward). Here, viewpoint numbers are assigned from left to right and from top to bottom. That is, the imaging device 301 is 1, the imaging device 302 is 2, the imaging device 303 is 3, the imaging device 304 is 4, the imaging device 305 is 5, and similarly, the imaging device 308 is 8.
[0072]
At this time, the control unit 102 in FIG. 1 specifies the presence / absence of reduction, the presence / absence of combination, 2D selection, the number of viewpoints M in the horizontal direction, the number of viewpoints N in the vertical direction, and the arrangement order of the images. Here, in the 2D selection, the viewpoint number of an image used for two-dimensional display may be specified, or may not be specified in particular. In the case of FIG. 9, the number of viewpoints is M = 4 and N = 2. Further, the arrangement order of the images is a sequence of viewpoint numbers, and it is possible to designate either the same arrangement as the viewpoint numbers (in the order of viewpoint numbers) or an arbitrary arrangement (in the arbitrary order).
[0073]
The image combining unit 101 in FIG. 1 arranges the images of the respective viewpoints in a grid pattern when the presence / absence of the combination input from the control unit 102 indicates “with combination”. Here, it is assumed that the number of images arranged in the horizontal direction and the number of images arranged in the vertical direction are the same as the installation method of the imaging apparatus. That is, M images are arranged in the horizontal direction and N images are arranged in the vertical direction. FIG. 11 shows an example of an image arrangement order corresponding to FIG. In FIG. 11, one square represents an image, and the number is a viewpoint number. FIG. 11A shows a case where the image arrangement order is the same as the viewpoint number, and FIG. 11B shows an example where the arrangement order is arbitrary.
[0074]
If the presence / absence of reduction input from the control unit 102 indicates “with reduction”, the input image of each viewpoint is reduced. The reduction ratio at this time is not fixed and is determined according to the number of viewpoints. That is, it is reduced to 1 / M in the horizontal direction and 1 / N in the vertical direction. In the case of FIG. 9, the reduction ratio is 1/4 in the horizontal direction and 1/2 in the vertical direction. An example of the result of combining by the image combining unit 101 at this time is shown in FIG. When the presence / absence of reduction indicates “no reduction”, the result is as shown in FIG. H and V are the number of horizontal pixels and the number of vertical lines of each viewpoint image before reduction. When the presence or absence of reduction indicates “with reduction”, the result is as shown in FIG. The size of the image after reduction is horizontal H pixels and vertical V lines, and is the same as the image size of each viewpoint before reduction. The arrangement of the images coincides with the installation method of the imaging apparatus, and four images are arranged in the horizontal direction and two images are arranged in the vertical direction. Here, the numbers attached to the images indicate viewpoint numbers. The images are arranged in ascending order of viewpoint numbers. The upper left is an image captured by the imaging unit 301 (viewpoint number 1), and the lower right is an image captured by the imaging unit 308 (viewpoint number 8). Here, the reduction ratio is described as being fixed, but these may be variable. If variable, the reduction ratio is recorded in 3D information.
[0075]
When the presence / absence of combination indicates “no combination”, the image combination unit 101 outputs the images of the respective viewpoints in the arrangement order of the images specified by the control unit 102. Therefore, the arrangement order of the images at this time indicates the order in which the images are stored.
[0076]
The 3D information creating unit 103 creates 3D information by formatting the number of viewpoints in the horizontal direction, the number of viewpoints in the vertical direction, and the arrangement order of the images in addition to the presence / absence of reduction, the presence / absence of combination, and 2D selection. An example of the 3D information at this time is shown in FIG. Here, the image order indicates that the arrangement of images is “viewpoint number order” or “arbitrary order”. Thereafter, a sequence of viewpoint numbers designated by the control unit 102 as the image arrangement order is recorded. In the example of FIG. 11B, the first viewpoint number is 2, and the order is 3, 1, 4, 6, 7, 5, 8 in the following. When the image order indicates “order of viewpoint number”, these viewpoint numbers may be omitted.
[0077]
When the multiplexing unit 104 multiplexes the 3D information and the image data but does not combine the images, as described above, the output order from the image combining unit 101 is the arrangement order of the images specified by the control unit 102. Therefore, the order in which the image data is recorded in the file is also the image arrangement order.
[0078]
In addition, when the presence / absence of combination indicates “no combination”, the image combining unit 101 always outputs the viewpoint number designated by 2D selection first when outputting the image of each viewpoint. Also good.
[0079]
Further, in the above, the way of assigning the viewpoint number is fixed and the arrangement order of the images can be arbitrarily changed. However, the arrangement order of the images may be fixed and the way of assigning the viewpoint numbers may be arbitrarily changed. . Furthermore, the installation method of the imaging device can be not only a lattice arrangement but also an arbitrary arrangement. In this case, a reference imaging device (with a viewpoint number of 1) is selected, and the position is expressed in a coordinate system with this as the origin. In the 3D information, the position coordinates of the imaging device at each viewpoint are recorded in the order of viewpoint numbers.
[0080]
When the multi-viewpoint image data multiplexed as described above is played back by the image data playback device 200 in FIG. 8, in order to display it as a three-dimensional image, two viewpoints are selected from the multi-viewpoint images. , Left eye image and right eye image. In the case of the image data recorded by the photographing apparatus of FIG. 9, it is possible to select from among 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, and the like arranged in the horizontal direction. Further, if the display is rotated by 90 °, it is possible to display it as a three-dimensional image even if a set of 1 and 5, 2, and 6, 3, and 7, and 4 and 8 is selected.
[0081]
When the display is switched to the two-dimensional display during the three-dimensional display, the image of the viewpoint number designated by the 2D selection is displayed. However, when the image is not being displayed, any one of the displayed images may be selected and displayed. The selection method is used for, for example, the one with the smallest viewpoint number, the largest one, the one closest to the viewpoint number specified by 2D selection in the installation of the imaging device or the layout of the combined image, and the left eye image. And those used for the right-eye image, and are not particularly limited here. When the 2D selection does not specify an image to be used for 2D display, the image to be displayed is selected by a predetermined method.
[0082]
In the above embodiment, the case where there are a plurality of image inputs has been described. However, the present invention is also applicable to a case where a stereo adapter is attached to a monocular imaging device. FIG. 13 is a diagram showing a configuration example of an image taken with a stereo adapter attached. FIG. 13A shows a case where the image is taken without the stereo adapter attached, and FIG. 13B shows a case where the image is taken with the stereo adapter of a type whose angle of view is halved. (C) shows an example in which the angle of view does not change and the image is captured with a stereo adapter of a type that can capture a narrow range. When the stereo adapter is attached, left and right images are captured on one screen, and “combined” is recorded in the 3D information as “presence / absence of coupling”. Further, in the example of FIG. 13B, since the left eye image and the right eye image are each taken in a state reduced by ½ in the horizontal direction, “with reduction” is recorded as the presence / absence of reduction. In the example of FIG. 13C, the area to be photographed becomes narrow, but since the image is not reduced, “no reduction” is recorded as the presence / absence of reduction.
[0083]
As described above, in a conventional reproducing apparatus that handles data created by various 3D image photographing methods in a unified manner and does not have a 3D image display function, it is possible to display a 2D image normally. Since it becomes possible, it can have versatility.
[0084]
<Second Embodiment>
In the second embodiment of the present invention, image data is encoded and then multiplexed into a predetermined format. FIG. 14 shows the configuration of the image data creation device 110 according to this embodiment. 14, the same parts as those in FIG. 1 are denoted by the same reference numerals. The image data creation device 110 includes an image combining unit 111, a control unit 102, and the like that create an image obtained by combining the images K adjacent to each other from the image 1 of multiple viewpoints (number of viewpoints K, where K is an integer of 2 or more). 3D information creation unit 112 that formats 3D information by formatting the presence / absence of reduction, image placement, 2D selection, number of viewpoints, image placement order, and presence / absence of inversion, encoding unit 113 that encodes image data, The multiplexing unit 104 is configured.
[0085]
The operation of the image creating apparatus 110 configured as described above will be described. The control unit 102 specifies the presence / absence of reduction, the presence / absence of combination, 2D selection, the number of viewpoints M in the horizontal direction, the number N of viewpoints in the vertical direction, and the arrangement order of the images. Since the specification method is the same as that of the first embodiment, description thereof is omitted here.
[0086]
The image combining unit 111 selects an arrangement method from the input image 1 to the image K when the presence / absence of the connection input from the control unit 102 indicates “with connection”. The arrangement method can be selected from three arrangements: horizontal arrangement in which the left and right images are arranged horizontally, vertical arrangement in which the left and right images are arranged up and down, and a grid arrangement in which both the horizontal and vertical directions are arranged.
[0087]
Here, the image arrangement method may or may not coincide with the imaging device installation method. When matching the image arrangement with the installation method of the imaging unit, the vertical arrangement is obtained when M = 1 and N ≧ 2, the horizontal arrangement is obtained when M ≧ 2 and N = 1, and the lattice arrangement is obtained otherwise. If they are not matched, when M = 1, N ≧ 2 or M ≧ 2, N = 1, either vertical arrangement or horizontal arrangement may be selected. When the presence / absence of reduction input from the control unit 102 indicates “with reduction”, the input image of each viewpoint is reduced. Since the reduction method is the same as that of the first embodiment, the description thereof is omitted here.
[0088]
Further, the image combining unit 111 inverts the image of each viewpoint in the horizontal direction and the vertical direction as necessary. By doing so, it is possible to improve the continuity of the boundary portion and reduce the decrease in encoding efficiency. The presence or absence of inversion is determined so that the discontinuity at the boundary portion of each viewpoint image is minimized. The method for obtaining the discontinuity of the boundary part is the sum of the absolute difference of the pixel values on the left and right of the boundary part for luminance only, color difference only, or both luminance and color difference, and the discontinuity. A method of determining whether or not the pixel value has changed on the left and right sides of the boundary by threshold processing, and making the total number of changed pixels discontinuity can be used. As the left and right pixels of the boundary portion, one pixel each on the left and right adjacent to the boundary portion may be used, or a pixel that is two pixels or more away from the boundary portion may be used.
[0089]
FIG. 15 is a diagram illustrating an example of a result of inversion processing on an image captured by the imaging apparatus of FIG. The numbers in the figure indicate the viewpoint number of each image. Here, an image taken at the viewpoint number K is referred to as an image K. For example, since the upper left image was taken at viewpoint number 1, it becomes image 1. In FIG. 15, image 1 and image 3 are not inverted, and image 2 and image 4 are inverted only in the horizontal direction. Image 5 to image 8 are inverted in the vertical direction, and image 6 and image 8 are inverted in the horizontal direction. As shown in FIG. 15, the border portion of the image has a similar pattern, and the continuity of the border portion can be increased.
[0090]
The selected image arrangement method and the presence / absence of inversion of each viewpoint image are output to the 3D information creation unit 112.
[0091]
The encoding unit 113 encodes the image data combined by the image combining unit 111. As an encoding method, an international standard method such as JPEG or JPEG2000 is used for a still image. In addition, international standard systems such as MPEG-1, MPEG-2, MPEG-4, etc. are used for moving images. When only intra-frame coding is used as the moving image coding, a method such as Motion JPEG may be used. The image encoding method is not limited to the above, and a non-standard method may be used.
[0092]
The 3D information creation unit 112 formats the arrangement method and inversion presence / absence of the image input from the image combination unit 111, the presence / absence of reduction specified by the control unit 102, the presence / absence of combination, 2D selection, the number of viewpoints, and the arrangement order. To generate 3D information necessary for displaying the image as a three-dimensional image. An example of the format is shown in FIG. In FIG. 16, the arrangement method is recorded instead of the presence / absence of coupling with respect to FIG. 10, and the presence / absence of inversion is added. Here, the arrangement method is “horizontal arrangement”, “vertical arrangement”, “lattice arrangement” (when the presence of coupling is “with coupling”) and “separation” (when the presence of coupling is “no coupling”). Although either one is represented, the presence / absence of coupling (with / without coupling) and the arrangement method (horizontal arrangement / vertical arrangement / lattice arrangement) may be recorded separately. Further, the presence or absence of horizontal inversion and the presence or absence of vertical inversion from the first to the Kth may be recorded in the order of image arrangement or in the order of viewpoint numbers. Further, when creating 3D information, the set value may be used as it is, or may be encoded by fixed length encoding or variable length encoding.
[0093]
Since the operation of the multiplexing unit 104 is the same as that of the first embodiment, description thereof is omitted here.
[0094]
Next, a playback device for displaying the image data created by the image data creation device 110 as a three-dimensional image will be described.
[0095]
FIG. 17 is a block diagram showing a configuration of an image data reproducing apparatus according to the second embodiment of the present invention. In FIG. 17, an image data reproduction apparatus 210 includes a demultiplexing unit 201, a 3D information analysis unit 211 that analyzes 3D information, a decoding unit 212 that decodes encoded data, and an image conversion that converts the decoded image data into a display format. Part 213.
[0096]
The operation of the image data reproducing apparatus 210 configured as described above will be described.
[0097]
Since the operation of the demultiplexing unit 201 is the same as that of the first embodiment, description thereof is omitted here.
[0098]
The 3D information analysis unit 211 analyzes the 3D information and extracts the presence / absence of reduction, the image layout method, the number of viewpoints, the presence / absence of inversion, and 2D selection. The decoding unit 212 decodes image data from the encoded data separated by the demultiplexing unit 201.
[0099]
The image conversion unit 213 converts the decoded image data into a display format according to the presence / absence of reduction, the image arrangement method, the number of viewpoints, and the presence / absence of inversion input by the 3D information analysis unit 211. Since the display format is the same as that of the first embodiment, description thereof is omitted here.
[0100]
In the above embodiment, the inversion direction is determined regardless of the arrangement direction, but the arrangement direction and the inversion direction may be matched. That is, in the horizontal arrangement, the image is inverted only in the horizontal direction, and in the vertical arrangement, the image is inverted only in the vertical direction.
[0101]
Further, in the above embodiment, the continuity of the boundary portion between adjacent images is increased in order to efficiently encode, but in the case of a still image, the following may be performed. That is, when the size of the image is not an integral multiple of the size of the block on which the DCT operation is performed, the pixel is complemented by copying one column of pixels on the right end or one line of the bottom end, so that the image size becomes an integral multiple of the block size. To be. In the case of a still image, coding can be efficiently performed by matching the block boundary where the DCT operation is performed with the boundary portion of the image. At this time, the presence / absence of complementation and the number of columns or lines of copied pixels are recorded in the 3D information instead of the presence / absence of inversion.
[0102]
As described above, according to the present embodiment, when a three-dimensional image is encoded, encoding is performed so that the continuity of the boundary portion when the multi-viewpoint images are adjacent to each other is increased. It becomes possible to encode.
[0103]
<Third Embodiment>
In the third embodiment of the present invention, storage information indicating the storage position of image data is added to the above-described management information. FIG. 18 shows the configuration of the image data creation device 120 according to this embodiment. In FIG. 18, the same parts as those in FIG. 14 are denoted by the same reference numerals. The image data creation device 120 includes an image combining unit 111, a control unit 102, a 3D information creation unit 121, a storage information creation unit 122 that creates storage information indicating a storage position of image data, an encoding unit 123, 3D information, and management information. And a multiplexing unit 124 that multiplexes the image data.
[0104]
Regarding the image data creation device 120 configured as described above, the operation when “no join” is indicated by the presence or absence of the join and a plurality of image data is recorded in one file will be described. Since the operation other than this case is the same as that of the second embodiment, a description thereof is omitted here.
[0105]
As described in the second embodiment, the image combining unit 111 outputs the image K from the input image 1 in the arrangement order input by the control unit 102. Therefore, the encoding unit 123 encodes the image data input in the arrangement order in a predetermined encoding format, and then outputs the encoded data size to the storage information creation unit 122.
[0106]
The 3D information creation unit 121 creates 3D information and outputs the arrangement order to the storage information creation unit 122.
[0107]
The storage information creation unit 122 creates storage information in a predetermined format by associating viewpoint numbers with addresses. An example of the format at this time is shown in FIG. Here, the address indicates the start position of the image data in the file or the header accompanying it. The storage information creation unit 122 calculates the address of the image data of each viewpoint by setting the initial value of the address to 0 and adding the data size input from the encoding unit 123. Since the order in which the data sizes are input is the arrangement order, the address corresponding to the nth viewpoint number is an addition value of the data sizes up to the (n-1) th.
[0108]
For example, a case will be described in which 2, 3, 1, 4, 6, 7, 5, 8 are designated as the arrangement order when an image captured by the imaging apparatus shown in FIG. 9 is recorded. The image data is encoded in this order by the encoding unit 123 and is output to the storage information creation unit 122 in the same order as the data size after encoding. Here, assuming that the encoded data size is 100 bytes for each image, 100 is input to the storage information creation unit 122 as many as the number of viewpoints, the address of image 2 is 0 bytes of the initial value, and the address of image 3 is 100. The address of byte and image 1 is 200 bytes, and similarly, the address of image 8 is 700 bytes. Accordingly, as shown in FIG. 22A, the storage information at this time is such that the first viewpoint number is 2, the second viewpoint number is 3, the eighth viewpoint number is 8, and the first address is 0. The second address is 100 bytes, and the eighth address is 700 bytes.
[0109]
When multiplexing the encoded image data, 3D information, and management information, the multiplexing unit 124 changes the stored address according to the file format to be used and the start position of the address. Hereinafter, the address changing method for both the existing format and the new format will be described.
[0110]
FIG. 20 is a diagram showing a storage example when the above-described existing format is used as the file format. As described above, in the existing format, 3D information and management information are recorded in a form in which the header portion is expanded. In FIG. 20, A to D indicate the start positions of addresses. A indicates the start of the file, B indicates the start position of the 3D information, C indicates the start position of the management information, and D indicates the start position of the first image data. Here, assuming that the start position of the address is A, the multiplexing unit 124 adds the data size of the header portion including 3D information and management information to the address calculated by the storage information creation unit 122. If this data size is 20 bytes, the address of the storage information in the above example is changed from the first address to 20 bytes and the second address to 120 bytes as shown in FIG. Thus, the eighth address is changed to 720 bytes. The start position of the address may be any of B, C, and D in addition to A.
[0111]
FIG. 21 is a diagram showing an example of storage when a new format is used as the file format. In FIG. 21, A to C indicate the start positions of addresses, A indicates the start of the file, B indicates the start position of the management information, and D indicates the start position of the first file header. As shown in FIG. 21, since a file header is added to each image data, the address is information indicating the start position of the file header. Therefore, the size of each file header and the data size of 3D information and management information are added to the address calculated in the storage information creation unit 122. Here, assuming that the start position of the address is A, the size of each file header is 10 bytes, and the data size of 3D information and management information is 15 bytes, the address of the stored information in the above example is shown in FIG. As shown, the first address is changed to 15 bytes, the second address is changed to 125 bytes, and the eighth address is changed to 785 bytes.
[0112]
Next, a reproduction device for displaying the image data created by the image data creation device 120 as a three-dimensional image will be described.
[0113]
FIG. 23 is a block diagram showing a configuration of an image data reproducing apparatus according to the third embodiment of the present invention. In FIG. 23, an image data reproduction device 220 displays a demultiplexing unit 221, a storage information analysis unit 222 that analyzes stored information, a 3D information analysis unit 211, a decoding unit 212 that decodes encoded data, and displays decoded image data. It comprises an image conversion unit 213 for converting into a format.
[0114]
The operation of the image data reproducing apparatus 220 configured as described above will be described.
[0115]
The demultiplexer 221 reads data in which image data, 3D information, and management information are multiplexed as described above from a file recorded in a recording device such as an IC memory, and separates 3D information and management information. Further, the storage information is separated from the management information.
[0116]
The storage information analysis unit 222 analyzes the storage information separated by the demultiplexing unit 221 and extracts an address indicating the start position of the image data of each viewpoint.
[0117]
Since the operation of the 3D information analysis unit 211 is the same as that of the second embodiment, description thereof is omitted here. When the number of viewpoints and the viewpoint number are extracted by the 3D information analysis unit 211, a combination of viewpoint numbers for selecting an image for three-dimensional display is designated by a control unit (not shown). At this time, the stored information analysis unit 222 outputs an address corresponding to the designated viewpoint number to the demultiplexing unit 221.
[0118]
The demultiplexing unit 221 reads the image data from the designated address and outputs it to the decoding unit 212. Since the subsequent operation is the same as that of the second embodiment, the description thereof is omitted here.
[0119]
In the above embodiment, the image data is recorded in units of viewpoint numbers. However, in the case of a moving image, the image data is composed of a plurality of frames, so that in addition to the storage example shown in FIG. Alternatively, recording may be performed in units of frames as shown in FIG. 24, and storage information may be recorded for each group of frames. Here, in the first storage information, an address is recorded for the first frame data from the first image to the Kth image. Similarly for the storage information thereafter, addresses are recorded for the frame data from the first image to the Kth image. The start position of the address is the same as in FIG. At this time, the i-th frame data from the first image to the K-th image (where i is an integer of 1 or more) is a frame to be displayed at the same timing, and a frame corresponding to a certain viewpoint number is displayed. If it does not exist, the storage information relating to the viewpoint number may be omitted. The storage information may be recorded with an address indicating the storage location of the next storage information. Further, the stored information may be recorded at a predetermined frame interval instead of being recorded at every frame interval. In this example, the storage information is not recorded in the management information. However, the first storage information or all the storage information may be included in the management information and recorded. FIG. 24 shows an example of an existing format file, but the same applies to a new format file.
[0120]
In the above embodiment, the address indicating the storage position of the image data is recorded as the storage information. However, the following may be used instead.
[0121]
In general, a unique code word (start code) for indicating the start position is added to the encoded image data. By retrieving the start code from the data, it is possible to access the target image data without decoding the encoded image data. However, the start code is generally different depending on the encoding method, and may not be added depending on the encoding method. Therefore, the multiplexing unit 124 in FIG. 18 records a start code unique to the multiplexing at the head of the image data. An example of storage at this time is shown in FIG. Furthermore, the start code may be recorded not only at the image data but also at the start position of the 3D information and management information, or the recording of the address indicating the start position and the recording of the start code may be used in combination.
[0122]
In the above embodiment, the order in which the addresses are recorded in the storage information is the same as the arrangement order. However, the addresses may be always recorded in the order of viewpoint numbers (ascending order or descending order), It may be in order.
[0123]
In the above embodiment, the address (absolute address) obtained by accumulating the data size is recorded as the storage information. However, the data size itself (relative address) may be recorded. In this way, it is possible to reduce the data size required when recording stored information.
[0124]
In the above embodiment, the case where image data is encoded has been described. However, the present invention may be applied to image data that is not encoded.
[0125]
As described above, according to the present embodiment, only necessary image data is read without reading all the image data recorded in the file, thereby efficiently performing file access and reducing the required memory amount. And the processing speed can be improved.
[0126]
【The invention's effect】
According to the present invention, in the image data creation device that creates image data in a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, the combination information indicating whether or not to combine images of different viewpoints An information generating means for generating is provided, and the data format includes the combined information, whereby an advantageous effect that image data for three-dimensional display can be provided with versatility can be obtained.
[0127]
According to the present invention, in the image data creation device that creates image data of a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, an image to be combined when images of different viewpoints are combined The information generation means for generating the arrangement information indicating the arrangement method of the image, and the data format includes the arrangement information, thereby providing the advantageous effect that the image data for three-dimensional display can have versatility. Is obtained.
[0128]
According to the present invention, in the image data creation device that creates image data of a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, the reduction information indicating whether or not to reduce the image. By providing reduced information generating means for generating, and the data format includes the reduced information, an advantageous effect that image data for three-dimensional display can be provided with versatility can be obtained.
[0129]
According to the present invention, in the image data creation device for creating image data of a predetermined data format from a plurality of images corresponding to each of the plurality of viewpoints, the selection information for selecting one of the plurality of viewpoints is provided. By including selection information generating means for generating, and the data format includes the selection information, an advantageous effect that image data for three-dimensional display can be provided with versatility is obtained.
[0130]
According to the present invention, in the image data creation device that creates image data in a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, viewpoint information generation that generates viewpoint information indicating the number of viewpoints And the data format includes the viewpoint information, so that an advantageous effect that image data for three-dimensional display can be provided with versatility can be obtained.
[0131]
Further, according to the present invention, the viewpoint information includes information indicating the number of viewpoints in the horizontal direction and / or information indicating the number of viewpoints in the vertical direction, so that the image data for three-dimensional display is versatile. The advantageous effect that it can be provided is obtained.
[0132]
According to the present invention, when the image is reduced at a predetermined reduction rate, the image data for three-dimensional display is made versatile by determining the predetermined reduction rate using the number of viewpoints. An advantageous effect that it can be obtained is obtained.
[0133]
According to the present invention, in the image data creation device for creating image data in a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, the sequence information indicating the order in which the plurality of images are stored is provided. An order information generating means for generating is provided, and the data format includes the order information, thereby providing an advantageous effect that image data for three-dimensional display can be provided with versatility.
[0134]
According to the present invention, when images from different viewpoints are combined, the order information indicates the order of the combined arrangement, whereby image data for three-dimensional display can be made versatile. An advantageous effect is obtained.
[0135]
According to the present invention, in the image data creation device for creating image data in a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, the inversion information indicating whether or not to invert the image is provided. Inversion information generation means for generating the image data, and the data format includes the inversion information so that the image data for three-dimensional display has versatility, and when the image is encoded, it is efficiently encoded. The advantageous effect of being able to do this is obtained.
[0136]
According to the present invention, the inversion information includes information indicating whether to invert the image in the horizontal direction and / or information indicating whether to invert the image in the vertical direction. When image data for dimensional display is provided with versatility and an image is encoded, an advantageous effect that the image data can be encoded efficiently is obtained.
[0137]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes combined information indicating whether or not images of different viewpoints are combined, and reproduces the plurality of images using the combined information, so that image data with versatility is displayed according to a display device. The advantageous effect that it can be displayed is obtained.
[0138]
Further, according to the present invention, when images from different viewpoints are combined, the analysis unit analyzes the arrangement information indicating the arrangement method of the combined images, thereby obtaining versatile image data. An advantageous effect is obtained that display can be performed according to the display device.
[0139]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes reduced information indicating whether or not the image has been reduced, and reproduces the plurality of images using the reduced information, thereby providing versatile image data according to a display device. The advantageous effect that it can be displayed is obtained.
[0140]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes selection information for selecting one of the plurality of viewpoints, and reproduces an image corresponding to one of the plurality of viewpoints using the selection information, thereby displaying versatile image data. An advantageous effect that the display can be performed according to the apparatus is obtained.
[0141]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes the viewpoint information indicating the number of viewpoints, and reproduces the plurality of images using the viewpoint information, so that versatile image data can be displayed according to a display device. Effects can be obtained.
[0142]
Further, according to the present invention, the viewpoint information includes information indicating the number of viewpoints in the horizontal direction and / or information indicating the number of viewpoints in the vertical direction, thereby displaying image data with versatility. It is possible to obtain an advantageous effect that display can be performed according to the above.
[0143]
In addition, according to the present invention, when the image is reduced at a predetermined reduction rate, image data with general versatility is displayed by determining the predetermined reduction rate using the number of viewpoints. An advantageous effect that the display can be performed according to the apparatus is obtained.
[0144]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes the order information indicating the order in which the plurality of images are stored, and reproduces the plurality of images using the order information, thereby displaying versatile image data in accordance with the display device. The advantageous effect of being able to be obtained is obtained.
[0145]
According to the present invention, when images from different viewpoints are combined, the order information indicates the order of the combined arrangement, thereby displaying image data with versatility according to the display device. The advantageous effect that it can be obtained.
[0146]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes inversion information indicating whether or not the image is inverted, and reproduces the plurality of images using the inversion information, thereby displaying image data with versatility according to a display device. The advantageous effect of being able to do this is obtained.
[0147]
According to the present invention, the inversion information includes information indicating whether the image is inverted in the horizontal direction and / or information indicating whether the image is inverted in the vertical direction. Thus, an advantageous effect that image data having versatility can be displayed according to a display device can be obtained.
[0148]
In addition, according to the present invention, in the image data creation device that creates image data in a predetermined data format from a plurality of images corresponding to each of a plurality of viewpoints, information generation that generates information indicating the head position of each image And the data format includes the information, thereby providing an advantageous effect that image data for three-dimensional display can have versatility and selectivity of an arbitrary image with high efficiency. .
[0149]
In addition, according to the present invention, by using the data length from a predetermined position in the image data to the head position as information indicating the head position of each image, versatility and efficiency can be added to the image data for three-dimensional display. An advantageous effect that it is possible to have good selectivity of any image is obtained.
[0150]
In addition, according to the present invention, by using the data size of each image as the information indicating the head position of each image, the image data for three-dimensional display has versatility and selectivity of an arbitrary image with high efficiency. An advantageous effect that it can be obtained is obtained.
[0151]
According to the present invention, in the image data reproduction apparatus for reproducing a plurality of images corresponding to each of a plurality of viewpoints from image data in a predetermined data format, the analysis means for analyzing the data format is provided, and the analysis The means analyzes the information indicating the head positions of a plurality of images, and reproduces an arbitrary image of the plurality of images using the information, thereby obtaining an image of an arbitrary viewpoint from the image data having versatility. An advantageous effect is obtained that it is possible to select efficiently and display according to the display device.
[0152]
In addition, according to the present invention, by using the data length from a predetermined position in the image data to the head position as information indicating the head position of each image, it is possible to select an arbitrary viewpoint from the image data having generality. An advantageous effect that an image can be efficiently selected and displayed according to a display device can be obtained.
[0153]
Further, according to the present invention, by using the data size of each image as information indicating the head position of each image, an image of an arbitrary viewpoint can be efficiently selected from image data having general versatility, and the display device can The advantageous effect of being able to display accordingly is obtained.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an image data creation device according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration example of 3D information.
FIG. 3 is a diagram illustrating an example of combining left and right images.
FIG. 4 is a diagram illustrating an example of combining left and right images.
FIG. 5 is a diagram showing a file format of multiplexed data.
FIG. 6 is a diagram illustrating a format example when image data is stored in an existing format file.
FIG. 7 is a diagram illustrating a format example when image data is stored in a new format file.
FIG. 8 is a diagram showing a configuration of an image data reproducing device according to the first embodiment of the present invention.
FIG. 9 is a diagram illustrating an installation example of an imaging apparatus in the case of multiple viewpoints.
FIG. 10 is a diagram illustrating a grid arrangement of multi-viewpoint images.
FIG. 11 is a diagram illustrating an example of how to assign viewpoint numbers.
FIG. 12 is a diagram illustrating a format example of 3D information.
FIG. 13 is a diagram for explaining a configuration of an image captured by a stereo adapter.
FIG. 14 is a diagram showing a configuration of an image data creation device according to a second embodiment of the present invention.
FIG. 15 is a diagram illustrating an example of inversion processing.
FIG. 16 is a diagram illustrating a format example of 3D information.
FIG. 17 is a diagram showing a configuration of an image data reproducing device according to a second embodiment of the present invention.
FIG. 18 is a diagram showing a configuration of an image data creation device according to a third embodiment of the present invention.
FIG. 19 is a diagram illustrating a format example of stored information.
FIG. 20 is a diagram illustrating a format example when image data is stored in an existing format file.
FIG. 21 is a diagram illustrating a format example when image data is stored in a new format file.
FIG. 22 is a diagram illustrating an example of stored information.
FIG. 23 is a diagram showing a configuration of an image data reproducing device according to a third embodiment of the present invention.
FIG. 24 is a diagram illustrating a format example when image data is stored in an existing format file.
FIG. 25 is a diagram illustrating a format example when image data is stored in an existing format file.
FIG. 26 is a diagram illustrating an image display format in a time division method.
FIG. 27 is a diagram for explaining the concept of the parallax barrier method;
FIG. 28 is a diagram for explaining an image display format in the parallax barrier method;
FIG. 29 is a diagram showing a composite image in a conventional method.
[Explanation of symbols]
100, 110, 120 Image data creation device
101, 111 Image combiner
102 Control unit
103, 112, 121 3D information creation unit
104, 124 Multiplexer
113, 123 Encoding unit
122 Stored information creation unit
200, 210, 220 Image data reproducing apparatus
201, 221 Demultiplexer
202, 211 3D information analysis unit
203, 213 Image converter
212 Decoding unit
222 Stored information analysis unit
301 to 306 Imaging device

Claims (2)

In an image data creation device for creating stereoscopic image data including a plurality of images corresponding to each of a plurality of viewpoints capable of stereoscopic viewing ,
The images used in the case of reproducing image data for the stereoscopic as a 2D image, the plurality of users from the viewpoint selects one, 2D selection information generating for generating 2D selection information indicating a selected viewpoint With means,
2. The image data creation apparatus according to claim 1, wherein the stereoscopic image data is obtained by multiplexing 2D selection information generated by the 2D selection information generation unit. Stereoscopic image data is reproduced based on a plurality of images corresponding to each of the plurality of viewpoints from stereoscopic image data including a plurality of images corresponding to each of the plurality of viewpoints capable of stereoscopic viewing. In an image data reproducing apparatus provided with an image reproducing means,
Wherein the stereoscopic image data, the image used in the case of reproducing a 2D image, and 2D selection information is multiplexed is information for selecting one from among the plurality of viewpoints,
The image reproduction means includes
Analyzing means for analyzing 2D selection information from the stereoscopic image data;
A selected image reproduction means for reproducing an image corresponding to the viewpoint selected by the 2D selection information as a 2D image ;
An image data reproducing apparatus comprising:

Images