JP2006180366A - Stereoscopic image generating apparatus and method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire stereoscopic images or two-dimensional images on the basis of an image desired by a user in a stereoscopic image generating apparatus such as a digital camera for generating the stereoscopic images. <P>SOLUTION: By the depression of a release button by a user, two image pickup parts provided in the digital camera respectively acquire images S1 and S2. Then, a selection image 70 is displayed on the LCD of the digital camera. In this case, the user selects, on the basis of either left and right image pickup parts, the two-dimensional image and the stereoscopic vision image are to be generated. In the case that the two-dimensional image is selected, the two-dimensional image is generated on the basis of the image selected by the user. In the case that the stereoscopic image is selected, the difference information of the two images is generated on the basis of the image selected by the user and the stereoscopic image is generated from the image as a reference and the difference information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a stereoscopic image generation apparatus and method for generating a stereoscopic image capable of combining a plurality of images and performing stereoscopic viewing using parallax, and a program for causing a computer to execute the stereoscopic image generation method It is about.

  Various methods for generating a stereoscopic image for expressing a planar image as a stereoscopic image are known. This stereoscopic image is realized by intentionally generating the parallax of both eyes generated by the interval between the right eye and the left eye. That is, a three-dimensional feeling that expresses a video by giving different images to the left and right eyes of the person viewing the image is expressed. Specifically, for example, in stereoscopic viewing by the naked-eye parallel method, two images corresponding to the left and right eyes are prepared, and the same object in the two images is shifted to the left and right for stereoscopic viewing. Is realized. This is due to the fact that the object far away from the viewer is almost in the same position when viewed with either left or right eye, but the near object is shifted to the left or right, that is, both eyes have parallax. .

  Various devices such as a mobile phone, a notebook computer, and a television equipped with a 3D liquid crystal for stereoscopic viewing are also known. In these apparatuses, a right-eye image and a left-eye image are generated, and a stereoscopic image obtained by synthesizing these two images is displayed on a 3D liquid crystal to express a stereoscopic image. is there.

  On the other hand, in order to generate a stereoscopic image, it is necessary to photograph the same subject from a plurality of different viewpoints. For this reason, various methods have been proposed for shooting for the purpose of generating a stereoscopic image. For example, 2 for generating a stereoscopic image by using a camera using two types of optical systems or attaching a stereoscopic image capturing adapter to the camera and capturing the same subject from two different viewpoints. A method of acquiring two images (see Patent Documents 1 and 2) has been proposed.

  In addition, it is possible to select whether to generate a stereoscopic image from a plurality of images acquired by a plurality of imaging devices or to generate an image (two-dimensional image) that is not a stereoscopic image, so that the stereoscopic imaging is performed by one imaging device. An apparatus has been proposed that performs both image generation and two-dimensional image generation (see Patent Document 3). According to this apparatus, a stereoscopic image and a two-dimensional image can be selectively obtained by a single photographing apparatus.

Further, in order to record a stereoscopic image while maintaining compatibility with a conventional system, although it is a moving image, difference information between a plurality of images acquired by a plurality of imaging devices is obtained, and at least of the plurality of images An apparatus that records one image and difference information has been proposed (see Patent Document 3). Here, since the correlation between the images acquired by the plurality of imaging devices is large, the information amount of the difference information is also small. Therefore, the file capacity of the stereoscopic image can be reduced.
Japanese Patent Laid-Open No. 7-134345 JP-A-10-191395 JP-A-10-90814 JP-A-6-30445

  However, in the method described in Patent Document 4, the user cannot select which of the plurality of images is used to generate the difference information. Further, the method described in Patent Document 4 can only generate a stereoscopic image, and selectively generates a stereoscopic image and a two-dimensional image like the method described in Patent Document 3. I can't.

  SUMMARY An advantage of some aspects of the invention is that a stereoscopic image or a two-dimensional image can be acquired based on an image desired by a user.

A stereoscopic image generation apparatus according to the present invention is a stereoscopic image generation apparatus for generating a stereoscopic image.
A plurality of photographing means for photographing the same subject from a plurality of viewpoints;
Which of the plurality of images acquired by the plurality of photographing means is used as a reference to generate a stereoscopic image, or which of the plurality of images is used as a reference to generate a two-dimensional image Selection accepting means for accepting selection;
Difference information generating means for generating difference information between the plurality of images when it is selected to generate the stereoscopic image;
When it is selected to generate the stereoscopic image, the stereoscopic image is generated from the image selected as the reference and the difference information, and when it is selected to generate the two-dimensional image, it is selected as the reference. And an image generation means for generating the two-dimensional image from the obtained image.

  Here, the “stereoscopic image” includes a reference image and difference information. When an image file of a stereoscopic image is opened, a plurality of images are reproduced from the reference image and difference information. These images are arranged side by side so that an image that can be viewed stereoscopically can be reproduced. A plurality of images are reproduced from the reference image and difference information, and these images can be combined into a single image so that they can be viewed stereoscopically when displayed on a 3D liquid crystal. Also good.

  The stereoscopic image generating apparatus according to the present invention may further include image recording means for recording the stereoscopic image and the two-dimensional image.

  In the stereoscopic image generating apparatus according to the present invention, the image generating means may be a means for generating a stereoscopic image including the image selected as the reference and the difference information as one file.

  In the stereoscopic image generating apparatus according to the present invention, the image recording unit may be a unit that generates a stereoscopic image including another image in which the image selected as the reference and the difference information are associated with each other.

A stereoscopic image generation method according to the present invention is a stereoscopic image generation method for generating a stereoscopic image.
Shoot the same subject from multiple viewpoints,
Which of the plurality of images acquired by the plurality of photographing means is used as a reference to generate a stereoscopic image, or which of the plurality of images is used as a reference to generate a two-dimensional image Accept the choice,
When it is selected that the stereoscopic image is generated, difference information between the plurality of images obtained by the plurality of photographing units is generated,
When it is selected to generate the stereoscopic image, the stereoscopic image is generated from the image selected as the reference and the difference information,
When it is selected to generate the two-dimensional image, the two-dimensional image is generated from the image selected as the reference.

  The stereoscopic image generation method according to the present invention may be provided as a program for causing a computer to execute the method.

  According to the present invention, a stereoscopic image is generated based on any of a plurality of images acquired by a plurality of photographing means, or a two-dimensional image is generated based on any of the plurality of images. Selection of whether to generate is accepted. And when it selects that a stereoscopic vision image is produced | generated, the difference information between several images is acquired. Then, when it is selected to generate a stereoscopic image, a stereoscopic image is generated from the image selected as a reference and difference information, and when it is selected to generate a two-dimensional image, it is selected as a reference. A two-dimensional image is generated from the image. For this reason, the user can generate and acquire a stereoscopic image or a two-dimensional image based on a desired image. In addition, since the acquired stereoscopic image includes a reference image and difference information having a smaller capacity than the reference image, the stereoscopic image is compared with a case where a plurality of images are acquired as they are as a stereoscopic image. File capacity can be reduced.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a digital camera to which a stereoscopic image generating apparatus according to an embodiment of the present invention is applied.

  As shown in FIG. 1, the digital camera 1 captures the same subject, forms a subject image on the light receiving surface, performs photoelectric conversion, and displays image data (images representing two images S1 ′ and S2 ′). The camera unit 6A, 6B each having a photographic lens, a CCD, a shutter, and the like, and the entire digital camera 1 are controlled and the sampling timing of the image data is output. CPU 22, which performs various controls such as control, image data recording control and display control, image size change processing, and image quality represented by image data such as sharpness correction, gamma correction, contrast correction and white balance correction Signal processing units 24A and 24B that perform setup processing to improve the A / D converters 26A and 26B that convert data into digital image data S1 and S2, a strobe 28, a strobe control unit 30 that controls the operation of the strobe 28, and various instructions are given to the digital camera 1. And an I / O 34 that is an interface for inputting setting information of the input unit 32 is provided.

  The camera unit 6A, the signal processing obtainment 24A, and the A / D conversion unit 26A are referred to as an imaging unit 20A, and the camera unit 6B, the signal processing obtainment 24B, and the A / D conversion unit 26B are referred to as an imaging unit 20B. Here, when the digital camera 1 is pointed at the subject, an imaging unit that captures the subject from the left side is denoted by 20A, and an imaging unit that captures the subject from the right side is denoted by 20B.

  In the input unit 32, the number of recording pixels and the recording format of the image data in the memory card 10 are set, the operation mode of the digital camera 1 such as the shooting mode and the playback mode is set, and the thumbnail image is set in the playback mode. Various switches such as a mode switching dial, a cross key, a setting switch, and a release button for switching between a list of images and an enlarged image display are provided.

  The digital camera 1 also includes a compression / decompression unit 36 that compresses image data by a technique typified by JPEG or motion JPEG, and a memory that is detachably attached to the memory card slot 38. A card interface (I / F) 42 for converting image data in order to record or read image data on the card 10 is provided.

  The digital camera 1 includes a system memory 44 including a program for operating the CPU 22 such as an image generation program to be described later, a ROM in which various constants are recorded, and a RAM serving as a work area when the CPU 22 executes processing. A rewritable nonvolatile memory 46 that can continue to record various setting constants of the digital camera 1 even after the power is shut off, a timer 48 that is driven at the time of timer shooting, and a calendar clock 50 that records the current time. An LCD control unit 54 that performs D / A conversion for displaying image data on a liquid crystal monitor (LCD) 16 and a VRAM that temporarily stores image data for display instructed by the CPU 22 The frame memory 56 and the code information commanded by the CPU 22 are converted into characters and message data for display. And OSD58 is provided for on-screen display with the image to. A battery 60 for supplying power to each unit is also provided.

  In addition, the digital camera 1 is provided with a difference calculation unit 62 that calculates difference information Ssub representing the difference between the images S1 and S2, as will be described later. Further, an image generation unit 64 for generating a stereoscopic image SR or a two-dimensional image SH from the images S1 and S2 is provided.

  The images S1 ′ and S2 ′ acquired by the camera units 6A and 6B are subjected to image processing by the signal processing units 24A and 24B, and then converted into digital images S1 and S2 by the A / D conversion units 26A and 26B. The Then, a stereoscopic image SR or a two-dimensional image SH having a format to be described later is generated from the images S1 and S2, and further compressed by the compression / decompression unit 36 and recorded in the memory card 10.

  As will be described later, the image generation unit 64 generates a stereoscopic image SR based on one of the images S1 and S2 acquired by the imaging units 20A and 20B. In addition, a two-dimensional image SH is generated based on one of the images S1 and S2. This process will be described later.

  Here, in the present embodiment, the stereoscopic image SR is generated based on which of the two images S1 and S2 at the time of shooting, or the two-dimensional image SH is based on which of the two images S1 and S2. Let the user choose whether to generate FIG. 2 is a diagram showing a selection screen displayed on the LCD 16 immediately after the user performs shooting by pressing the release button.

  As shown in FIG. 2, the selection screen 70 includes a “2D (left camera)” command 70a that is selected when the two-dimensional image SH is generated based on the image S1 acquired by the imaging unit 20A, and the imaging unit 20B. The “2D (right camera)” command 70b selected when generating the two-dimensional image SH with the acquired image S2 as a reference, and the image S1 acquired by the imaging unit 20A among the imaging units 20A and 20B as a three-dimensional image When generating the stereoscopic image SR with reference to the “3D (base: left)” command 70c selected when generating the visual image SR and the image S2 acquired by the imaging unit 20B among the imaging units 20A and 20B. A “3D (base: right)” command 70d to be selected is displayed. Each command can be selected by moving the cursor using the input unit 32 and highlighting it.

  Next, processing for generating the stereoscopic image SR will be described. When the user selects the command 70c or the command 70d on the selection screen 70 described above, the images S1 and S2 are input to the difference calculation unit 62.

  The difference calculation unit 62 determines a reference image according to which of the commands 70c and 70d is selected by the user, and calculates difference information Ssub between the reference image and the other image. That is, the difference information Ssub is calculated based on the image S1 when the user selects the command 70c, and based on the image S2 when the user selects the command 70d. Hereinafter, calculation of the difference information Ssub using the image S1 as a reference will be described.

  First, the difference calculation unit 62 calculates a difference value between corresponding pixels of the image S1 and the image S2. Specifically, the difference value is calculated by subtracting the pixel of the image S1 from the pixel of the image S2. Then, the calculated difference value is output as Ssub. Here, since the images S1 and S2 are taken of the same subject, the correlation between the images is very high, and the information amount of the difference value is significantly smaller than that of the original image.

  Further, the difference information Ssub may be calculated using a technique used for moving image compression. In this method, the images S1 and S2 are divided into blocks (for example, 16 pixels × 16 pixels), and information (motion vector) indicating to which position the subject included in the image S1 has moved in the image S2 in units of blocks. This motion vector is used as difference information Ssub. Also in this method, the difference information Ssub is significantly smaller than the original image.

  The image generation unit 64 generates the stereoscopic image SR from the reference image (here, S1) and the difference information Ssub. When generation of a two-dimensional image is selected, a two-dimensional image SH is generated from the reference image S1 or S2.

  FIG. 3 is a diagram illustrating a format of the stereoscopic image SR generated by the image generation unit 64. As shown in FIG. 3, the image file 100 of the stereoscopic image SR includes header information 101, an additional information pointer 102, base image data 103 representing the reference image S <b> 1, and difference information data representing the difference information Ssub. 104.

  The header information 101 includes Exif information such as the shooting date and the number of pixels.

  When the image file 100 represents an image of the stereoscopic image SR, the image file 100 always includes the difference information data 104 at the address ahead of the address of the base image data 103. Therefore, the additional information pointer 102 indicates the head address of the difference information data 104.

  FIG. 4 is a diagram showing the format of the two-dimensional image SH. As shown in FIG. 4, the image file 110 of the two-dimensional image SH includes header information 111, an additional information pointer 112, and base image data 113 representing the reference image S1.

  The header information 111 includes Exif information such as the shooting date and the number of pixels.

  When the image file 110 represents the image of the two-dimensional image SH, the difference information data 104 is not included in the address ahead of the address of the base image data 113 unlike the stereoscopic image SR. Therefore, information “NULL” is written in the additional information pointer 112.

  Therefore, it is possible to identify whether the image file is the stereoscopic image SR or the two-dimensional image SH according to the information described in the additional information pointer of the image file.

  Here, the two-dimensional image SH and the stereoscopic image SR obtained in the present embodiment will be described. FIG. 5 is a view for explaining photographing using the digital camera 1 according to the present embodiment. As shown in FIG. 5, it is assumed here that the subject H has been photographed.

  FIG. 6 is a diagram illustrating images acquired by the imaging units 20A and 20B. FIG. 6A illustrates an image S1 acquired by the imaging unit 20A, and FIG. 6B illustrates an image acquired by the imaging unit 20B. S2 is shown respectively. When the user selects the command 70c on the selection screen 70, the image generation unit 64 generates the difference information Ssub from the images S1 and S2 with the image S1 as a reference, and the stereoscopic view in the format shown in FIG. 3 from the image S1 and the difference information Ssub. An image SR is generated. In the case of generating the two-dimensional image SH, when the command 70a is selected, the two-dimensional image SH having the format shown in FIG. 4 is generated on the basis of the image S1 (see FIG. 6C). In this case, the two-dimensional image SH is the image S1 itself.

  Note that the generated stereoscopic image SR and two-dimensional image SH are recorded in the memory card 10.

  Next, processing performed in the present invention will be described. FIG. 7 is a flowchart showing processing performed in the present embodiment. When the user presses the release button to shoot the subject, the CPU 22 starts processing. First, the imaging units 20A and 20B acquire images S1 and S2 (step ST1). Next, the selection screen 70 is displayed on the LCD 16 (step ST2), and the selection of a command by the user is accepted (step ST3).

  When the user selects the commands 70a and 70b (that is, generation of a two-dimensional image), the image generation unit 64 uses the image S1 or S2 as a reference in accordance with the command selected by the user, and the two-dimensional format shown in FIG. An image SH is generated (step ST4). The two-dimensional image SH is recorded on the memory card 10 (step ST5), and the process ends.

  On the other hand, when the user selects commands 70c and 70d (that is, generation of a stereoscopic image), the difference calculation unit 62 generates difference information Ssub based on the image S1 or S2 according to the command selected by the user ( In step ST6, the image generation unit 64 generates a stereoscopic image SR having the format shown in FIG. 3 based on the image S1 or S2 in accordance with the command selected by the user (step ST7). The stereoscopic image SR is recorded on the memory card 10 (step ST5), and the process is terminated.

  When an instruction to reproduce the stereoscopic image SR is given, the image S1 is reproduced from the base image data 103 included in the image file 100 (when the image S1 is used as a reference), and the base image data 103 and the difference information data 104 are used. The image S2 is reproduced. Then, by arranging the reproduced images S1 and S2 side by side, the images that can be stereoscopically viewed are reproduced. Note that the reproduced images S1 and S2 may be combined into one image and reproduced so that stereoscopic viewing is possible when displayed on the 3D liquid crystal.

  As described above, in the present embodiment, of the images S1 and S2 acquired by the imaging units 20A and 20B, on the selection screen 70, which image 70c and 70d is used as a reference to generate the stereoscopic image SR, Alternatively, when the selection of which of the images S1 and S2 is used to generate the two-dimensional image SH with reference to the commands 70a and 70b and the generation of the stereoscopic image SR is selected, Difference information Ssub is generated, and a stereoscopic image SR is generated from the images S1 and S2 serving as a reference. In addition, when it is selected to generate the two-dimensional image SH, the two-dimensional image SH is generated from the images S1 and S2 serving as the reference. Therefore, the user can generate the stereoscopic image SR or the two-dimensional image SH with reference to the desired image. In addition, since the generated stereoscopic image SR includes either one of the images S1 and S2 and the difference information Ssub, the file capacity of the stereoscopic image is reduced compared to a case where a plurality of images are generated as they are as a stereoscopic image. can do.

  Moreover, the capacity of the memory card 10 can be saved by recording the stereoscopic image as the image file 100 including any one of the images S1 and S2 and the difference information Ssub.

  In the above-described embodiment, a stereoscopic image is generated in a format including a reference image and difference information Ssub as a single image file. However, as shown in FIG. 8, two images associated with each other are generated. Each of the files 120 and 130 may include base image data and difference information data. Here, the image file 120 includes header information 121 and base image data 122. The image file 130 includes header information 131 and difference information data 132. In order to associate the image files 120 and 130, the header information 121 describes the file name of the image file 130, and the header information 131 describes the file name of the image file 120 as the file name of the associated file.

  Moreover, in the said embodiment, although the digital camera 1 is provided with the two imaging parts 20A and 20B, you may provide one imaging part so that removal is possible. FIG. 9 is a front view showing a configuration of the digital camera 2 in which one of the imaging units is detachably provided. As shown in FIG. 9, the digital camera 2 includes a main body 2A and a sub photographing unit 2B. One imaging unit is provided in the main body 2A, and the other imaging unit is provided in the sub-imaging unit 2B.

  Such a digital camera 2A can be used as a normal digital camera without using the sub photographing unit 2B.

  Also, as shown in FIG. 10, by attaching the sub photographing unit 2B to the right side of the digital camera 2A, it functions as a digital camera capable of generating a stereoscopic image as in the above embodiment. Further, as shown in FIG. 11, the sub photographing unit 2B can be attached to the left side of the digital camera 2A.

  When the sub photographing unit 2B can be attached to both the left and right sides of the main body 2A, the imaging unit provided in the main body 2A is often superior in function to the sub photographing unit 2B. It is preferable that an image acquired by the imaging unit of the main body 2A is always used as a reference when generating a two-dimensional image, and an image acquired by the imaging unit of the main body 2A is always used as a reference when generating a stereoscopic image. In this case, the user can select a reference image depending on whether the sub photographing unit 2B is attached to the left or right position of the main body 2A.

1 is a schematic block diagram showing a configuration of a digital camera to which a stereoscopic image generating apparatus according to an embodiment of the present invention is applied. Diagram showing selection screen Diagram showing the format of a stereoscopic image Diagram showing 2D image format The figure for demonstrating imaging | photography using the digital camera by this embodiment. The figure which shows the image acquired by imaging part 20A, 20B A flowchart showing processing performed in the present embodiment The figure which shows the format at the time of making a stereoscopic vision image into two image files linked | related with each other Front view showing a configuration of a digital camera in which one imaging unit is detachably provided The figure which shows the state where the sub photographing section is attached to the right side of the main body The figure which shows the state where the sub photographing section is attached to the left side of the main body

Explanation of symbols

1, 2 Digital camera 2A Main unit 2B Sub-shooting unit 16 LCD
20A, 20B Imaging unit 22 CPU
32 Input unit 44 System memory 54 LCD control unit 60 Battery 62 Difference calculation unit 64 Image generation unit 70 Selection screen 100, 110, 120, 130 Image file

Claims (6)

In a stereoscopic image generation device for generating a stereoscopic image,
A plurality of photographing means for photographing the same subject from a plurality of viewpoints;
Which of the plurality of images acquired by the plurality of photographing means is used as a reference to generate a stereoscopic image, or which of the plurality of images is used as a reference to generate a two-dimensional image Selection accepting means for accepting selection;
Difference information generating means for generating difference information between the plurality of images when it is selected to generate the stereoscopic image;
When it is selected to generate the stereoscopic image, the stereoscopic image is generated from the image selected as the reference and the difference information, and when it is selected to generate the two-dimensional image, it is selected as the reference. A stereoscopic image generation apparatus comprising: an image generation unit configured to generate the two-dimensional image from the obtained image.   The stereoscopic image generating apparatus according to claim 1, further comprising image recording means for recording the stereoscopic image and the two-dimensional image.   3. The stereoscopic image generation apparatus according to claim 2, wherein the image generation means is a means for generating a stereoscopic image including the image selected as the reference and the difference information as one file.   3. The stereoscopic image generation apparatus according to claim 2, wherein the image recording unit is a unit that generates a stereoscopic image including another file in which the image selected as the reference and the difference information are associated with each other. . In a stereoscopic image generation method for generating a stereoscopic image,
Shoot the same subject from multiple viewpoints,
Which of the plurality of images acquired by the plurality of photographing means is used as a reference to generate a stereoscopic image, or which of the plurality of images is used as a reference to generate a two-dimensional image Accept the choice,
When it is selected to generate the stereoscopic image, the difference information between the plurality of images is generated,
When it is selected to generate the stereoscopic image, the stereoscopic image is generated from the image selected as the reference and the difference information,
A stereoscopic image generation method, characterized in that, when it is selected to generate the two-dimensional image, the two-dimensional image is generated from the image selected as the reference. In a program for causing a computer to execute a stereoscopic image generation method for generating a stereoscopic image,
Procedures for shooting the same subject from multiple viewpoints;
Which of the plurality of images acquired by the plurality of photographing means is used as a reference to generate a stereoscopic image, or which of the plurality of images is used as a reference to generate a two-dimensional image A procedure to accept the selection,
A procedure for generating difference information between the plurality of images when it is selected to generate the stereoscopic image;
A procedure for generating the stereoscopic image from the image selected as the reference and the difference information when it is selected to generate the stereoscopic image;
And a procedure for generating the two-dimensional image from the image selected as the reference when it is selected to generate the two-dimensional image.

Images