JP2005321987A - Image formation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image formation method capable of providing an intermediate image (processing image) suitable for encoding processing (compression) or the like. <P>SOLUTION: In two digit sequences (m, n) on the upper stage side of each picture element or pixel, m is a visual point number and n is an arrangement number (sub-visual point image number). Two digit sequences (x, y) on the lower stage side are coordinates of picture elements or pixels. Picture elements of both images are extracted in a checkered pattern, respectively, and the arrangements of picture elements in both the visual point images are in an interpolating relation. In the right eye image, for example, the thick actual line frame is a first sub-visual point image, and the thick dotted line frame is a second sub-visual point image. In the left eye image, also, the first sub-visual point image and the second sub-visual point images are present. In the intermediate image, the picture elements constituting each sub-visual point image are combined while keeping the vertical and horizontal arrangement relations. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image generation method suitable for stereoscopic images.

  Conventionally, a parallax barrier method, a lenticular lens method, and the like are known as methods for realizing stereoscopic image display without requiring special glasses, but these methods are for the right eye having binocular parallax. Video and left-eye video are displayed alternately on the screen, for example, in the form of vertical stripes, and this display video is separated by a parallax barrier, lenticular lens, etc., and guided to the viewer's right and left eyes respectively. It is what makes you see. Further, Patent Document 1 below is known as an oblique barrier system in which a plurality of pinholes are arranged obliquely.

FIG. 22 shows the processing contents when generating an oblique barrier composite image (stereoscopic image) using the right-eye (R) image and the left-eye (L) image. In each image, an R (red) pixel, a G (green) pixel, and a B (blue) pixel are arranged side by side. For example, one R pixel having a coordinate “00” and one coordinate “00” are arranged. The G pixel and one B pixel having the coordinate “00” constitute one picture element (pixel) having the coordinate “00”. In this process, a composite image is obtained by alternately extracting and arranging pixels from the right eye image and the left eye image in units of pixels. FIG. 23 shows a conventional process for generating an intermediate image from a composite image.
Japanese Patent No. 3096613

  However, in the intermediate image shown in FIG. 23, since the right eye pixels are collected and the left eye pixels are gathered, although there is less degradation in image quality than when the synthesized image is compressed as it is, it is adjacent in the intermediate image. Since the picture elements to be processed are picture elements arranged obliquely on the original viewpoint image, the compression of the intermediate image cannot sufficiently prevent the deterioration of the image quality.

  In view of the above circumstances, an object of the present invention is to provide an image generation method capable of obtaining an intermediate image (processing image) suitable for performing encoding processing (compression).

  In order to solve the above-described problem, the image generation method of the present invention can be used to generate an oblique barrier-type stereoscopic image using N (N is an integer of 2 or more) viewpoint images. A method is characterized in that N sub-viewpoint images are extracted in a form that is obliquely shifted in each viewpoint image and each picture element is arranged in the horizontal direction and the vertical direction. With this configuration, N sub-viewpoint images are extracted for each viewpoint image, and for example, an intermediate image or a stereoscopic image can be generated using these N sub-viewpoint images.

  The image generation method of the present invention is an image generation method for generating an intermediate image that can be used for generating an oblique barrier type stereoscopic image using N (N is an integer of 2 or more) viewpoint images. , N sub-viewpoint images that are obliquely shifted in each viewpoint image and in which each picture element is arranged in the horizontal direction and the vertical direction are extracted, and the horizontal direction of the picture elements constituting each sub-viewpoint image is extracted. An intermediate image in which these picture elements are gathered is generated while maintaining the alignment and the vertical alignment. An intermediate image may be generated using all of a plurality of color pixels constituting each picture element in each sub-viewpoint image.

  In these image generation methods, each viewpoint image is created using computer graphics technology, and for each viewpoint image, rotation processing around the viewpoint of the camera coordinate system, movement of the view volume, and viewport conversion are performed. The sub-viewpoint image may be generated by performing at least one process of moving the origin.

  In addition, the image generation method of the present invention extracts each viewpoint image from the oblique barrier type stereoscopic image generated based on N (N is an integer of 2 or more) viewpoint images, Extracts N sub-viewpoint images in which pixel elements with the same color pixel arrangement are aligned in the horizontal and vertical directions, and maintains the horizontal and vertical alignments of the picture elements that make up each sub-viewpoint image However, an intermediate image in which these picture elements are gathered is generated.

  With these configurations, the picture elements that make up the intermediate image maintain the relationship between the picture elements arranged in the horizontal and vertical directions in the viewpoint image, and the correlation between the picture elements is high, and compression processing is performed. It will be suitable for. An entire intermediate image obtained by collecting N intermediate images obtained from each sub-viewpoint image is also suitable for the compression process. The intermediate image or the entire intermediate image may be compressed.

  As described above, according to the present invention, by focusing on the sub-viewpoint image in which the picture elements are aligned in the horizontal and vertical directions for the viewpoint image or the stereoscopic image used in the oblique barrier method, the encoding process ( Even if compression is performed, it is possible to generate an intermediate image that can reduce image quality degradation.

  Hereinafter, an image generation method according to an embodiment of the present invention will be described with reference to FIGS.

  FIG. 1A shows a pixel arrangement (stereoscopic image) of a two-eye oblique barrier method. In the figure, “1” is the first viewpoint image, and “2” is the second viewpoint image. R is a red pixel (or red data), G is a green pixel (or green data), and B is a blue pixel (or blue data). In the oblique barrier method, pixels from the same viewpoint are arranged in an oblique direction.

  FIG. 1B shows only the first viewpoint image extracted from the stereoscopic image. R, G, and B exist in each thick solid line frame and each thick dotted line frame in the figure. A group of R, G, and B is called a picture element. The thick solid line frames (picture elements) have the same arrangement (arrangement) of R, G, and B, and are arranged in the vertical direction and the horizontal direction. Similarly, the thick dotted line frames (picture elements) have the same arrangement (arrangement) of R, G, and B, and are arranged in the vertical direction and the horizontal direction. From the stereoscopic image, it is possible to generate a sub-viewpoint image by a group of thick solid line frames (picture elements) and a sub-viewpoint image by a group of thick dotted line frames (picture elements). That is, since the number of viewpoints is 2, two sub-viewpoint images are generated.

  FIG. 1C shows another example of sub-viewpoint image extraction. R, G, and B exist in each thick solid line frame and each thick dotted line frame in the figure. The thick solid line frames (picture elements) have the same arrangement (arrangement) of R, G, and B, and are arranged in the vertical direction and the horizontal direction. Similarly, the thick dotted line frames (picture elements) have the same arrangement (arrangement) of R, G, and B, and are arranged in the vertical direction and the horizontal direction.

  A plurality of viewpoint images with different viewpoints can be obtained, for example, by imaging a subject with a plurality of cameras provided at a distance between eyes. In addition, a plurality of 2D video data having different viewpoints can be obtained by setting a plurality of line-of-sight directions to a 3D image of a polygon image and projecting the respective planes by computer graphics technology.

  FIG. 2 shows a process for generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image. It is explanatory drawing which showed. In the figure, in two numerical strings (m, n) on the upper side of each picture element or pixel, m is a viewpoint number, and n is an arrangement number (sub-viewpoint image number). Further, the two numeric strings (x, y) on the lower side are the coordinates of the picture element or pixel. In both images, picture elements are extracted in a checkered pattern, and the arrangement of picture elements in both viewpoint images is in an interpolating relationship. Taking the right eye image as an example, the thick solid line frame is the first sub-viewpoint image, and the thick dotted line frame is the second sub-viewpoint image. Similarly, the first sub-viewpoint image and the second sub-viewpoint image exist in the left-eye image.

  In the intermediate image, the picture elements constituting each sub-viewpoint image are gathered while maintaining the vertical and horizontal alignment relationships. Further, these four intermediate images may be collected to generate one image (entire intermediate image). It is preferable to perform encoding processing (compression) using the intermediate image or the entire intermediate image. Pixels in the picture elements of both sub-viewpoint images are arranged in the order of R, B, and G in the horizontal direction.

  In the composite image, the pixels in the picture elements of each sub-viewpoint image are arranged every other pixel in the horizontal direction, and are arranged in the order of R, B, and G in the horizontal direction. In the synthesized pixel, an image of one horizontal line is formed only from the first sub-viewpoint image in both viewpoint images, and the horizontal one line below is composed of only the second sub-viewpoint image in both viewpoint images. In FIG. 2, in the synthesized image, the order in which RGB are arranged is the same in all of the viewpoint image and the sub-viewpoint image. In addition, the blank portion on the end side in the composite image is a pixel missing (data nonexistent) portion.

  FIG. 3 generates a right eye (R) image and a left eye (L) image, generates an intermediate image from these two viewpoint images, and generates a composite image (stereoscopic image) from the intermediate image. It is explanatory drawing which showed the example of a process. In both images, picture elements are extracted in a checkered pattern, and the arrangement of the picture elements in both images is the same (non-interpolation relationship). Taking the image for the right eye as an example, the solid line frame becomes the first sub-viewpoint image, and the dotted line frame becomes the second sub-viewpoint image. Similarly, the first sub-viewpoint image and the second sub-viewpoint image exist in the left-eye image.

  In the intermediate image, the picture elements constituting each sub-viewpoint image are gathered while maintaining the vertical and horizontal alignment relationships. Further, these four intermediate images may be collected to generate one image (entire intermediate image). It is preferable to perform encoding processing (compression) using the intermediate image or the entire intermediate image.

  For the right-eye image of the composite image, the pixel elements in the sub-viewpoint image are arranged in the order of R, B, and G in the horizontal direction. For the left-eye image, the pixel elements in the sub-viewpoint image are arranged. The pixels are arranged in the order of G, R, and B in the horizontal direction. Further, in the composite image, the pixels of the picture elements in each sub-viewpoint image are arranged every other pixel in the horizontal direction, and are arranged in the order of R, B, G or G, R, B in the horizontal direction. Further, in the composite image, an image of one horizontal line is formed only from the first sub-viewpoint image in both viewpoint images, and the horizontal one line below is composed of only the second sub-viewpoint image in both viewpoint images. In FIG. 3, the order of RGB in the viewpoint image is the same in the composite image. In addition, the blank portion on the end side in the composite image is a pixel missing (data nonexistent) portion.

  FIG. 4 shows another example of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image. It is explanatory drawing which showed the example of a process. In both images, picture elements are extracted in a checkered pattern, and the arrangement of picture elements in both images is in an interpolating relationship. Taking the image for the right eye as an example, the solid line frame becomes the first sub-viewpoint image, and the dotted line frame becomes the second sub-viewpoint image. Similarly, the first sub-viewpoint image and the second sub-viewpoint image exist in the left-eye image.

  In the intermediate image, the picture elements constituting each sub-viewpoint image are gathered while maintaining the vertical and horizontal alignment relationships. Further, these four intermediate images may be collected to generate one image (entire intermediate image). It is preferable to perform encoding processing (compression) using the intermediate image or the entire intermediate image.

  In the composite image, picture element pixels in both sub-viewpoint images are arranged in a V shape in the order of R, G, and B in the horizontal direction. That is, the G pixel among the R, G, and B pixels is moved downward by one pixel. In the composite image, a vertical pixel line is formed by the first sub-viewpoint image in the right-eye image and the second sub-viewpoint image in the left-eye image, and the vertical pixel line located next to the first sub-viewpoint image is the right-eye image. A second sub-viewpoint image in the image for use and a first sub-viewpoint image in the image for the left eye. In FIG. 4, in the synthesized image, the order in which RGB are arranged is the same in all of the viewpoint image and the sub-viewpoint image. In addition, the blank portion on the end side in the composite image is a pixel missing (data nonexistent) portion.

  FIG. 5 shows another example of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image. It is explanatory drawing which showed the example of a process. In both images, picture elements are extracted in a checkered pattern, and the arrangement of the picture elements in both images is the same (non-interpolation relationship). Taking the image for the right eye as an example, the solid line frame becomes the first sub-viewpoint image, and the dotted line frame becomes the second sub-viewpoint image. Similarly, the first sub-viewpoint image and the second sub-viewpoint image exist in the left-eye image.

  In the intermediate image, the picture elements constituting each sub-viewpoint image are gathered while maintaining the vertical and horizontal alignment relationships. Further, these four intermediate images may be collected to generate one image (entire intermediate image). It is preferable to perform encoding processing (compression) using the intermediate image or the entire intermediate image.

  In the composite image, picture element pixels in both sub-viewpoint images are arranged in a V shape in the order of R, G, and B in the horizontal direction. That is, the G pixel among the R, G, and B pixels is moved downward by one pixel. On the other hand, the pixels in the picture element of the left-eye image are arranged in a Λ shape. That is, the G pixel of the R, G, and B pixels is moved up by one pixel. In the synthesized image, a vertical pixel line is formed by the first sub-viewpoint image in the right-eye image and the first sub-viewpoint image in the left-eye image, and the vertical pixel line located next to the first sub-viewpoint image is the right-eye image. A second sub-viewpoint image in the image for use and a second sub-viewpoint image in the image for the left eye. In FIG. 5, the order of RGB in the viewpoint image is the same in the composite image. The blank portion on the end side in the composite image is a pixel missing portion (data does not exist).

  The file of the composite image (stereoscopic image) generated by the above-described processing includes, for example, information on the number of viewpoints, information on the arrangement of the sub-viewpoint images (arrangement about pixel colors, coordinates of picture elements (pixels)). It is preferable to give information such as arranging every other pixel in the horizontal direction, arranging in a V shape, and arranging in a Λ shape. Alternatively, information such as the first method and the second method may be simply given, and the contents of each method may be held in the image processing apparatus (software).

  FIG. 6 is an explanatory diagram showing a processing example for generating an intermediate image from a composite image (stereoscopic image). In FIG. 6, the picture elements of each viewpoint image and sub-viewpoint image are selected so that the order in which RGB are arranged is the same. In the figure, in two numerical strings (m, n) on the upper side of each picture element or pixel, m is a viewpoint number, and n is an arrangement number (sub-image number). Further, the two numeric strings (x, y) on the lower side are the coordinates of the picture element or pixel. Based on the information recorded in the composite image (stereoscopic image) file, the image processing device (software), for example, information on the number of viewpoints, information on the arrangement of the sub-viewpoint images (arrangement of pixel colors, pictures The information about the arrangement of the coordinates of the elements (pixels), every other pixel in the horizontal direction, the arrangement in a V shape, the arrangement in a Λ shape, etc. is obtained. In the processing example illustrated in FIG. 6, the image processing apparatus (software) has an odd-numbered horizontal line for the synthesized image, the pixel of the first sub-viewpoint image of the right-eye image and the pixel of the first sub-viewpoint image of the left-eye image. Are alternately arranged, and the picture element of the first sub-viewpoint image of the right-eye image (having the same coordinates) and the picture element of the first sub-viewpoint image of the left-eye image (having the same coordinates) Stuff). Similarly, with respect to the composite image, the even-numbered horizontal line knows that the pixels of the second sub-viewpoint image of the right-eye image and the pixels of the second sub-viewpoint image of the left-eye image are alternately arranged, A picture element (having the same coordinates) of the second sub-viewpoint image of the right-eye image and a picture element (having the same coordinates) of the second sub-viewpoint image of the left-eye image are extracted. Further, the image processing apparatus (software) knows that the picture elements extracted from the composite image (having the same coordinates) have the pixel arrangement of R, B, and G, and each viewpoint of the intermediate image Processing is performed so that R, G, and B are arranged in the image. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships. The intermediate images composed of the four sub-viewpoint images may be combined and handled as one image (entire intermediate image). The same applies to the following processing examples. It is preferable to perform encoding processing (compression) on the intermediate image or the entire intermediate image.

  FIG. 7 is an explanatory diagram showing another example of processing for generating an intermediate image from a composite image (stereoscopic image). In FIG. 7, the arrangement of RGB of the viewpoint images of the composite image is the same. The image processing apparatus (software) obtains various information from the file of the composite image (stereoscopic image). In the processing example of FIG. 7, the image processing apparatus (software) of the composite image has odd-numbered horizontal lines that include pixels of the first sub-viewpoint image of the right-eye image and pixels of the first sub-viewpoint image of the left-eye image. Knowing that they are arranged alternately, the first sub-viewpoint image element of the right eye image (having the same coordinates) and the first sub-viewpoint image element of the left eye image (having the same coordinates) ) And extract. Similarly, with respect to the composite image, the even-numbered horizontal line knows that the pixels of the second sub-viewpoint image of the right-eye image and the pixels of the second sub-viewpoint image of the left-eye image are alternately arranged, A picture element (having the same coordinates) of the second sub-viewpoint image of the right-eye image and a picture element (having the same coordinates) of the second sub-viewpoint image of the left-eye image are extracted. Further, the image processing apparatus (software) has the pixel arrangement extracted from the composite image (having the same coordinates) with the pixel arrangement of R, B, and G for the right eye image and the pixel arrangement for the left eye image. It is known that the arrangement is G, R, B, and processing is performed so that the arrangement is R, G, B on each viewpoint image of the intermediate image. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships.

  In the example of FIG. 7, since there is an unused area at the left end (right end) in the composite image, a dummy pixel column may be added. Alternatively, as in the example of FIG. 6 described above, the sub-viewpoint image may be enlarged.

  FIG. 8 is an explanatory diagram showing another example of processing for generating an intermediate image from a composite image (stereoscopic image). In FIG. 8, the arrangement of RGB differs between the sub-viewpoint images of the composite image. The image processing apparatus (software) obtains various information from the file of the composite image (stereoscopic image). In the processing example of FIG. 8, the image processing apparatus (software) has the odd-numbered horizontal lines of the composite image and the pixels of the first sub-viewpoint image of the right eye image and the pixels of the first sub-viewpoint image of the left eye image. Are alternately arranged, and the picture element of the first sub-viewpoint image of the right-eye image (having the same coordinates) and the picture element of the first sub-viewpoint image of the left-eye image (having the same coordinates) Stuff). Similarly, with respect to the composite image, the even-numbered horizontal line knows that the pixels of the second sub-viewpoint image of the right-eye image and the pixels of the second sub-viewpoint image of the left-eye image are alternately arranged, A picture element (having the same coordinates) of the second sub-viewpoint image of the right-eye image and a picture element (having the same coordinates) of the second sub-viewpoint image of the left-eye image are extracted. Further, the image processing apparatus (software) has a pixel arrangement (with the same coordinates) extracted from the composite image in which the pixel arrangement is R for the first viewpoint image of the right eye image and the second viewpoint image of the left eye image. , B, and G, the second viewpoint image of the right eye image and the first viewpoint image of the left eye image know that the pixel arrangement is G, R, and B. Are processed so as to form an array of R, G, and B respectively. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships.

  FIG. 9 is an explanatory diagram showing another example of processing for generating an intermediate image from a composite image (stereoscopic image). In FIG. 9, the picture elements of each viewpoint image and sub-viewpoint image are selected so that the order in which RGB are arranged is the same. The image processing apparatus (software) obtains various types of information from the composite image (stereoscopic image) file. In the processing example of FIG. 9, the image processing apparatus (software) knows that the picture elements (those having the same coordinates) are arranged in a V shape in both the left eye image and the right eye image in the composite image. The pixels are extracted in a V shape. Further, the image processing apparatus (software) processes the picture elements (having the same coordinates) extracted from the composite image so that they are arranged in R, G, B on each viewpoint image of the intermediate image. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships.

  FIG. 10 is an explanatory diagram showing another example of processing for generating an intermediate image from a composite image (stereoscopic image). In FIG. 10, the RGB arrangement of the viewpoint images of the composite image is the same. The image processing apparatus (software) obtains various information from the file of the composite image (stereoscopic image). In the processing example of FIG. 10, the image processing apparatus (software) knows that the picture elements of the right-eye image (those having the same coordinates) are arranged in a V shape in the composite image, and the V shape is obtained. The pixels are extracted to the left eye, and it is learned that the picture elements of the left-eye image (those having the same coordinates) are arranged in a Λ shape, and the pixels are extracted in a Λ shape. Further, the image processing apparatus (software) processes the picture elements (having the same coordinates) extracted from the composite image so that they are arranged in R, G, B on each viewpoint image of the intermediate image. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships.

  FIG. 11 is an explanatory diagram showing another example of processing for generating an intermediate image from a composite image (stereoscopic image). In FIG. 11, the RGB arrangement differs between the sub-viewpoint images of the composite image. The image processing apparatus (software) obtains various information from the file of the composite image (stereoscopic image). In the processing example of FIG. 11, the image processing apparatus (software) is arranged in a Λ shape with a picture element of the right eye image (having the same coordinates) arranged in a V shape in the composite image. Knowing that there is something, pixels are extracted in a V shape or Λ shape, and the picture elements of the left eye image (those with the same coordinates) are arranged in a Λ shape or V shape Knowing that there is something, extract the pixels in Λ shape or V shape. Further, the image processing apparatus (software) processes the picture elements (having the same coordinates) extracted from the composite image so that they are arranged in R, G, B on each viewpoint image of the intermediate image. Then, an intermediate image is obtained by collecting the pixel groups of each sub-viewpoint image while maintaining the vertical and horizontal alignment relationships.

  It is not essential to obtain various types of information from a composite image (stereoscopic image) file. An image of pixel thinning synthesis (see FIGS. 22 and 23) can also be used. For example, in FIG. 23, the thick solid line picture elements in the composite image (the middle pixel is replaced with the middle pixel of the adjacent picture element to match the viewpoint) are arranged in the horizontal and vertical directions. The arrangement is maintained while maintaining the arrangement relationship, and this is used as the first sub-viewpoint image. In addition, the thick dotted line picture elements in the composite image (the middle pixel is replaced with the middle pixel of the adjacent picture element to match the viewpoint) are arranged in the horizontal and vertical directions. This is maintained and collected, and this is used as the second sub-viewpoint image. In the first sub-viewpoint image, the picture elements are gathered to form an intermediate image, and similarly, in the second sub-viewpoint picture, the picture elements are gathered to form an intermediate image.

  FIGS. 12A and 12B are explanatory diagrams showing processing for generating a sub-viewpoint image from each viewpoint image. FIG. 12 shows only the first viewpoint image extracted from the composite image. The first sub-viewpoint image in the first viewpoint image is realized by extracting pixels in an array of R, B, and G (pixels whose coordinates match) with respect to odd-numbered horizontal lines. In addition, the second sub-viewpoint image in the first viewpoint image is realized by extracting pixels in an array of R, B, and G (pixels having the same coordinates) with respect to even-numbered horizontal lines. This concept can be applied to the process of extracting the sub-viewpoint image of each viewpoint image in the composite image and the process of extracting the sub-viewpoint image of each viewpoint image generated by live action or computer graphics. Specific processing when using computer graphics is described below.

  FIGS. 13A and 13B are explanatory diagrams showing a process of generating a sub-viewpoint image from each viewpoint image by rotating the viewpoint in the computer graphics camera coordinate system as a rotation center. The rotation amount may be performed at the same ratio as the pixel offset amount (see FIG. 12).

  FIGS. 14A and 14B are explanatory diagrams showing processing for generating a sub-viewpoint image from each viewpoint image by translating the view volume by computer graphics. The origin may be translated at the same ratio as the pixel offset.

  FIGS. 15A and 15B are explanatory diagrams showing a process of generating a sub-viewpoint image from each viewpoint image constituting a composite image by performing parallel movement of the origin at the time of viewport conversion in computer graphics. is there. The origin may be translated at the same ratio as the pixel offset.

  FIG. 16 is an explanatory diagram showing a pixel array of a composite image (stereoscopic video) of four viewpoints. The numerical values in FIG. 16 represent viewpoint numbers. In this figure, three pixels arranged diagonally are treated as one picture element. In this case, the central pixel of each picture element is a representative point. The positional relationship between the representative point and each pixel differs depending on the row. Also, the arrangement of pixels (R, B, G) in each picture element varies. FIG. 17 is an explanatory diagram showing the pixel arrangement of the first viewpoint image among the four viewpoint composite images. 18 shows the arrangement of the picture elements of the first sub-viewpoint image in the first viewpoint image with a thick line frame, and FIG. 19 shows the arrangement of the picture elements of the second sub-viewpoint image in the first viewpoint image. FIG. 20 shows the arrangement of picture elements of the third sub-viewpoint image in the first viewpoint image with a thick line frame, and FIG. 21 shows the arrangement of picture elements of the fourth sub-viewpoint image in the first viewpoint image. Is indicated by a bold frame. In either case, the picture elements in the sub-viewpoint image are aligned in the horizontal and vertical directions. The sub-viewpoint images are shifted from each other. In FIG. 18 to FIG. 21, the number of picture elements in the sub-viewpoint image is different because the coordinates of the picture elements are different between the sub-images. It is easier to process by matching the number of picture elements. In the image generating apparatus, rendering is performed four times to extract four sub-viewpoint images. However, since the number of pixels for each rendering is small, the processing speed is hardly reduced.

  Even in the case of these four viewpoints, it is preferable to generate an intermediate image by collecting picture elements in each sub-viewpoint image. Alternatively, the entire intermediate image may be generated by collecting four intermediate images at each viewpoint (four). It is preferable to perform an encoding process (compression) on the intermediate image and the entire intermediate image.

FIG. 1A is an explanatory diagram showing a pixel arrangement (stereoscopic image) of a binocular oblique barrier method, and FIG. 1B shows only the first viewpoint image in the stereoscopic image. It is explanatory drawing extracted and shown, and FIG.1 (c) is explanatory drawing which showed the other example of sub viewpoint image extraction. A process of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image is shown. It is explanatory drawing. A process of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image is shown. It is explanatory drawing. A process of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image is shown. It is explanatory drawing. A process of generating a right-eye (R) image and a left-eye (L) image, generating an intermediate image from these two viewpoint images, and generating a composite image (stereoscopic image) from the intermediate image is shown. It is explanatory drawing. It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). It is explanatory drawing which showed the process example which produces | generates an intermediate image from a synthesized image (stereoscopic image). FIGS. 9A and 9B are explanatory diagrams showing processing for generating a sub-viewpoint image from the viewpoint images constituting the composite image. FIGS. 7A and 7B are explanatory diagrams showing processing for generating a sub-viewpoint image from each viewpoint image by rotating or translating a viewpoint in a computer graphics camera coordinate system. FIGS. 9A and 9B are explanatory diagrams showing processing for generating a sub-viewpoint image from each viewpoint image by translating the view volume by computer graphics. FIGS. 7A and 7B are explanatory diagrams showing processing for generating a sub-viewpoint image from each viewpoint image constituting a composite image by performing parallel movement of the origin at the time of viewport conversion in computer graphics. is there. It is explanatory drawing which showed the pixel arrangement | sequence of the synthesized image (stereoscopic image | video) of 4 viewpoints. It is explanatory drawing which showed the pixel arrangement | sequence of the 1st viewpoint image among the composite images of 4 viewpoints. It is explanatory drawing which showed the arrangement | sequence of the picture element of the 1st sub viewpoint image among 1st viewpoint images with the thick line frame. It is explanatory drawing which showed the arrangement | sequence of the pixel of the 2nd sub viewpoint image among 1st viewpoint images with the thick line frame. It is explanatory drawing which showed the arrangement | sequence of the picture element of the 3rd sub viewpoint image among 1st viewpoint images with the thick line frame. It is explanatory drawing which showed the arrangement | sequence of the pixel of the 4th sub viewpoint image among 1st viewpoint images with the thick line frame. It is explanatory drawing which showed the content of the conventional process which produces | generates the synthetic | combination image (stereoscopic image) of a diagonal barrier system using the image for right eyes (R) and the image for left eyes (L). It is a figure which shows a prior art example, Comprising: It is explanatory drawing which showed the content of the conversion of a synthesized image (stereoscopic image) and an intermediate image (processing image).

Explanation of symbols

R Right pixel (right picture element)
L Left pixel (left picture element)

Claims (7)

An image generation method that can be used to generate a stereoscopic image of an oblique barrier method using N (N is an integer of 2 or more) viewpoint images, and is configured to be obliquely shifted in each viewpoint image and An image generation method comprising extracting N sub-viewpoint images in which each picture element is arranged in a horizontal direction and a vertical direction. An image generation method for generating an intermediate image that can be used for generating an oblique barrier-type stereoscopic image using N (N is an integer of 2 or more) viewpoint images, and is shifted obliquely in each viewpoint image N sub-viewpoint images in which the picture elements are arranged in the horizontal direction and the vertical direction are extracted, and the horizontal arrangement and the vertical arrangement of the picture elements constituting each sub-viewpoint image are maintained. An image generation method characterized by generating an intermediate image in which these picture elements are gathered. 3. The image generation method according to claim 2, wherein an intermediate image is generated by using all of a plurality of color pixels constituting each picture element in each sub-viewpoint image. 4. The image generation method according to claim 1, wherein each viewpoint image is generated using a computer graphics technique, and the viewpoint image is rotated around the viewpoint of the camera coordinate system. A sub-viewpoint image is generated by performing at least one of processing, movement of a view volume, and movement of an origin at the time of viewport conversion. Each viewpoint image is extracted from an oblique barrier type stereoscopic image generated based on N (N is an integer of 2 or more) viewpoint images, and the color pixels in each viewpoint image have the same arrangement. N sub-viewpoint images that are arranged in the horizontal direction and the vertical direction, and an intermediate that combines these picture elements while maintaining the horizontal and vertical arrangements of the picture elements constituting each sub-viewpoint image An image generation method characterized by generating an image. 6. The image generation method according to claim 2, wherein the entire intermediate image is generated by collecting N intermediate images obtained from each sub-viewpoint image. 7. The image generation method according to claim 2, wherein the intermediate image or the entire intermediate image is compressed.

Images