JP2012070103A - Imaging device and image communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device or the like capable of easily generating a composite image with a small feeling of strangeness, and which is obtained by composing a subject image capable of being displayed three-dimensionally with a background image.SOLUTION: An imaging device has: an imaging part; a display that can display a 3-dimentional image based on two images; an image generator that generates two photographed images to be the basis for a 3-dimentional image, which are corresponding to a right eye and a left eye, based on image data generated by the imaging part; an image clipping part that clips images representing a main subject, which are obtained by removing a background from the two photographed images, respectively to acquire two main subject images corresponding to the right eye and the left eye; an image composition part that pastes the two main subject images clipped by the image clipping part on the background image to generate two composite images corresponding to the right eye and the left eye; a touch panel; and a position adjustment part that adjusts a disparity by changing a relative position of the main subject images in the two composite images depending on an input to the touch panel.

Description

  The present invention relates to an imaging apparatus that captures and displays an image, and an image communication system that includes the imaging apparatus and performs image communication according to a predetermined communication standard.

  In recent years, by acquiring a plurality of image data for the same subject and using the parallax of the subject included in the acquired plurality of image data, a three-dimensional image (hereinafter, referred to as a three-dimensional image) that can be viewed three-dimensionally by a user. An imaging apparatus such as a digital stereo camera that captures “3D images”) is becoming widespread. Recently, digital cameras that can shoot 3D movies and televisions that can view 3D images have also appeared.

  On the other hand, various devices for creating composite photographs have been developed. For example, Patent Document 1 discloses a photographic game machine that displays a background image, a photographic image of a person, etc., and a frame image in an overlapping manner. In addition, digital cameras that create composite photos of moving images based on continuously shot images have also been developed.

JP 2008-65112 A

  By the way, when a subject image such as a person capable of 3D display is combined with another background image, a stereoscopic effect such as a sense of popping out or withdrawing of the subject image does not match the background image. May be natural.

  The present invention has been made in view of the above, and is an image pickup apparatus that can easily generate a 3D composite image that is a composite image obtained by combining a subject image that can be displayed in 3D with a background image, and that has a little uncomfortable feeling. An object of the present invention is to provide an image communication system that includes such an imaging device and performs image communication.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging unit that images a subject and generates image data, and a display unit that can display a three-dimensional image based on the two images. An image generation unit that generates two captured images corresponding to the right eye and the left eye, which are the basis of a three-dimensional image, based on the image data generated by the imaging unit, and a background is removed from the two captured images. An image extraction unit that acquires two main subject images corresponding to the right eye and the left eye by extracting each image representing the main subject, and the two main subject images extracted by the image extraction unit with a predetermined background An image composition unit that generates two composite images corresponding to the right eye and the left eye by pasting to an image, an input reception unit that receives an operation input from the outside, and the input reception Depending on input received by, characterized in that it comprises a position adjusting unit for adjusting the parallax by changing the relative position of the two main object image in the two composite images.

  In the imaging apparatus, the main subject is an image located in front of or in the center of the visual field in the two captured images.

  In the imaging apparatus, the main subject is an image having the largest parallax in the two captured images.

  The imaging apparatus further includes a light emitting unit that illuminates a field area of the imaging unit, and the main subject is an image of an area where reflected light is the strongest when the light emitting unit emits light. .

  In the imaging apparatus, the image composition unit generates the two composite images in which a background is displayed as a two-dimensional image by pasting the two main subject images on the same background image. .

  In the imaging apparatus, the image composition unit can display the background as a three-dimensional image by pasting the two main subject images on two background images in which one end in the horizontal direction of the visual field overlaps each other. The two synthesized images are generated.

  The image pickup device acquires two display screen shot images corresponding to the right eye and the left eye from a shot image generated by performing high-speed continuous shooting on a display screen that displays a three-dimensional image by a frame sequential method. The image composition unit generates the two composite images by pasting the two main subject images on the two display screen captured images acquired by the image acquisition unit, respectively. And

  The imaging device is configured so that when a three-dimensional image based on the two display screen captured images acquired by the image acquisition unit is displayed, the three-dimensional image appears to be virtually retracted from the display screen. It is further characterized by further comprising a left and right image discriminating unit for discriminating correspondence between the right eye and the left eye of the two display screen shot images.

  In the imaging device, either one of a display of a three-dimensional image based on two images and a display of a two-dimensional image obtained by superimposing the two images is displayed on the display unit according to a signal input from the outside. It further includes a display control unit for switching and displaying.

  In the imaging apparatus, the input receiving unit is a touch panel that is provided on the display screen of the display unit and receives an input of a signal according to the position of an object in contact with the outside, and the position adjusting unit is connected to the touch panel. The parallax of the main subject image is adjusted according to the trajectory of the contacted object.

  In the imaging apparatus, the position adjustment unit changes, in a two-dimensional image obtained by superimposing the two composite images, an overlapping region where one end in the left-right direction of the two main subject images overlaps according to the locus. Thus, the parallax is adjusted.

  In the imaging apparatus, the position adjustment unit further moves the two main subject images according to the locus while maintaining a relative positional relationship in a two-dimensional image obtained by superimposing the two composite images. By doing so, the position of the main subject image with respect to the background image is moved.

  In the imaging apparatus, the position adjustment unit further rotates the two main subject images according to the locus while maintaining a relative positional relationship in a two-dimensional image obtained by superimposing the two composite images. By doing so, the main subject image is rotated in the plane of the background image.

  The imaging apparatus uses the two images to generate a transmission image generation unit that generates a transmission image to be transmitted for display on the external device, and transmits the transmission image generated by the transmission image generation unit. And a communication unit for transmitting to an external device.

  In the imaging apparatus, the imaging unit includes two optical systems that respectively capture the two images.

  In the imaging apparatus, the imaging unit has a function of continuously capturing images at a predetermined interval while the imaging device is moving, and the image generation unit is an image continuously captured by the imaging unit. The two images are extracted from the image.

  In the imaging apparatus, the display control unit performs control to output two image data respectively corresponding to the two images to the display unit alternately arranged for each pixel in the horizontal direction on the display screen. Features.

  An image communication system according to the present invention is a display that displays a three-dimensional image based on the image for transmission received from the image pickup apparatus, which is communicably connected to the image pickup apparatus and the image pickup apparatus in accordance with a predetermined communication standard. And a device.

  According to the present invention, since the parallax of the main subject image pasted on the background image can be easily adjusted, it is possible to reduce a sense of incongruity between the background and the main subject image.

FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a schematic diagram illustrating a schematic configuration of a display unit included in the imaging apparatus illustrated in FIG. 1. FIG. 3 is a schematic diagram illustrating a situation when the imaging unit included in the imaging apparatus illustrated in FIG. 1 generates two pieces of image data in which one end in the left-right direction of each field of view overlaps. FIG. 4 is a diagram illustrating an example of an image corresponding to two image data in which the imaging unit has one end in the horizontal direction of each other's visual field overlapping each other in the situation illustrated in FIG. 3. FIG. 5 is a diagram illustrating an example of an image obtained by virtually superimposing the right eye image and the left eye image generated by the imaging unit under the situation illustrated in FIG. FIG. 6 is a diagram illustrating a first example of a method for extracting a main subject image from the right eye image and the left eye image generated by the imaging unit under the situation illustrated in FIG. FIG. 7 is a diagram illustrating a second example of a method for extracting a main subject image from the right eye image and the left eye image generated by the imaging unit under the situation illustrated in FIG. FIG. 8 is a diagram illustrating a third example of a method for extracting the main subject image from the right eye image and the left eye image generated by the imaging unit under the situation illustrated in FIG. FIG. 9 is a diagram illustrating an example of a template image. FIG. 10 is a diagram illustrating an example of a composite image in which a main subject image is pasted on a template image. FIG. 11 is a schematic diagram illustrating a state in which the parallax of the main subject image is adjusted (the pop-out distance is increased) on the superimposed image of the right-eye synthesized image and the left-eye synthesized image. FIG. 12 is a schematic diagram showing how the position of the main subject image is adjusted on the superimposed image of the right-eye synthesized image and the left-eye synthesized image. FIG. 13 is a schematic diagram illustrating a state in which the parallax of the main subject image is adjusted (the retract distance is increased) on the superimposed image of the right-eye synthesized image and the left-eye synthesized image. FIG. 14 is a flowchart showing the operation of the imaging apparatus shown in FIG. FIG. 15 is a diagram illustrating an example of a template image selection screen. FIG. 16 is a schematic diagram showing how the main subject is photographed. FIG. 17 is a flowchart showing an operation in the reproduction display process of the imaging apparatus shown in FIG. FIG. 18 is a schematic diagram illustrating a back side of an imaging apparatus that displays a superimposed image of a right-eye synthesized image and a left-eye synthesized image. FIG. 19 is a schematic diagram showing how the parallax of the main subject image is adjusted on the superimposed image of the right-eye synthesized image and the left-eye synthesized image. FIG. 20 is a schematic diagram showing how the parallax of the main subject image is adjusted on the superimposed image of the right-eye synthesized image and the left-eye synthesized image. FIG. 21 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 2 of the present invention. FIG. 22 is a schematic diagram illustrating a state in which a television display screen is captured by the imaging device illustrated in FIG. FIG. 23 is a diagram for describing a method for capturing an image from an image displayed on a television set into an imaging apparatus. FIG. 24 is a diagram illustrating a method for discriminating between the right eye image and the left eye image. FIG. 25 is a flowchart showing the operation of the imaging apparatus shown in FIG. FIG. 26 is a schematic diagram showing a configuration of an image communication system according to Embodiment 3 of the present invention. FIG. 27 is a block diagram illustrating a configuration of the imaging apparatus illustrated in FIG.

  DESCRIPTION OF EMBODIMENTS Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an imaging apparatus 1 according to Embodiment 1 of the present invention. As shown in FIG. 1, an imaging device 1 captures an image from two different positions and generates image data corresponding to two images in which the left and right ends of each field of view overlap each other, and the imaging device A posture detection unit 3 that detects one posture, an operation input unit 4 that receives input of various types of information of the imaging apparatus 1, a clock 5 that has a shooting date determination function and a timer function, and a two-dimensional image (hereinafter referred to as “2D”). A display unit 6 for displaying a 3D image, a touch panel 7 for receiving an input of a signal according to an external contact position or locus, and various types of information including image data generated by the imaging unit 2 are stored. A storage unit 8 and a control unit 9 that controls the operation of the imaging apparatus 1 are provided.

  The imaging unit 2 includes a first imaging unit 21, a second imaging unit 22, and a light emitting unit (strobe) 23 having different optical systems. The first imaging unit 21 and the second imaging unit 22 are arranged side by side on the same plane so that their optical axes L1 and L2 are parallel or have a predetermined angle.

  The first imaging unit 21 includes a lens unit 21a, a lens driving unit 21b, a diaphragm 21c, a diaphragm driving unit 21d, a shutter 21e, a shutter driving unit 21f, an imaging element 21g, and a signal processing unit 21h. .

  The lens unit 21a is configured by a focus lens, a zoom lens, or the like, and collects light from a predetermined visual field region. The lens driving unit 21b is configured by a DC motor or the like, and changes the focus position and focal length of the lens unit 21a by moving the focus lens, the zoom lens, and the like of the lens unit 21a on the optical axis L1.

  The diaphragm 21c adjusts the exposure by limiting the amount of incident light collected by the lens unit 21a. The aperture drive unit 21d is configured by a stepping motor or the like, and drives the aperture 21c.

  The shutter 21e sets the state of the image sensor 21g to an exposure state or a light shielding state. The shutter drive unit 21f is configured by a stepping motor or the like, and drives the shutter 21e according to the release signal.

  The imaging element 21g is realized by a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like that receives the light collected by the lens unit 21a and converts it into an electrical signal (analog signal), and converts the converted electrical signal. It outputs to the signal processor 21h.

  The signal processing unit 21h performs signal processing such as amplification on the electrical signal output from the image sensor 21g, and then converts the digital signal to digital image data by performing A / D conversion, and outputs the digital image data to the control unit 9.

The second imaging unit 22 is realized by the same configuration as the first imaging unit 21, and includes a lens unit 22a, a lens driving unit 22b, a diaphragm 22c, a diaphragm driving unit 22d, a shutter 22e, and a shutter driving unit 22f. The image sensor 22g and the signal processor 22h are included.
The light emitting unit 23 illuminates the field areas of the first imaging unit 21 and the second imaging unit 22.

  The posture detection unit 3 is configured by an acceleration sensor, and detects the posture of the imaging device 1 by detecting the acceleration of the imaging device 1. Specifically, the posture detection unit 3 detects the posture of the imaging device 1 with respect to the horizontal plane.

  The operation input unit 4 includes a power switch 41 that switches the power state of the imaging device 1 to an on state or an off state, a release switch 42 that inputs a release signal that gives a still image shooting instruction, and various shooting modes of the imaging device 1. A change-over switch 43 for switching various settings, a zoom switch 44 for performing a zoom operation of the imaging unit 2, and a 2D / 3D change-over switch 45 for switching the display mode on the display unit 6 between 2D image use and 3D image use.

  The timepiece 5 generates a time signal that serves as a reference for the operation of the imaging apparatus 1. Thereby, the control part 9 can set the acquisition time of image data, the exposure time of the image pick-up elements 21g and 22g, etc. The clock 5 also has a function as a timer for measuring time.

FIG. 2 is a schematic diagram illustrating a schematic configuration of the display unit 6. As shown in FIG. 1, the display unit 6 includes a backlight 61, a display panel 62, and a parallax barrier 63. The backlight 61 is configured by an LED (Light Emitting Diode) or the like, and irradiates light for displaying an image from the back side. The display panel 62 is configured by a display panel such as liquid crystal or organic EL (Electro Luminescence). The parallax barrier 63 is made of liquid crystal or the like and is laminated on the upper surface of the display panel 62. The parallax barrier 63 is provided with slits at an interval narrower than the interval between the pixels of the display panel 62, and separates images corresponding to the user's right eye E _R and left eye E _L , respectively. As such a parallax barrier 63, for example, a parallax barrier method is applied. Instead of the parallax barrier 63, a lens sheet in which lenticular lenses are stacked may be provided on the upper surface of the display panel 62.

In the display unit 6 having the above configuration, when 3D image data is input from the control unit 9, the display panel 62 performs horizontal alignment of the right eye image and the left eye image from the left end pixel in the horizontal direction under the control of the control unit 9. The images are alternately displayed in order, and the parallax barrier 63 separates the light emitted from each pixel of the display panel 62. For this reason, the right eye image reaches only the right eye E _R and the left eye image reaches only the left eye E _L. Thereby, the user can stereoscopically view the 3D image displayed on the display unit 6. Further, when the display unit 6 switches the display mode from the 3D image to the 2D image, the voltage applied to the parallax barrier 63 changes from the on state to the off state, so that the parallax barrier 63 transitions from the light shielding state to the transmissive state. Then, either the right eye image or the left eye image is output to the display panel 62.

  The touch panel 7 is provided so as to overlap the display screen of the display unit 6. The touch panel 7 functions as an input receiving unit that detects a position (or touched) where the user has touched (touched) based on information or an image displayed on the display unit 6 and receives an input of an operation corresponding to the contact position or locus. To do. In general, the touch panel includes a resistance film method, a capacitance method, an optical method, and the like. In the first embodiment, any type of touch panel is applicable.

  The storage unit 8 stores a captured image data storage unit 81 that stores image data captured by the imaging unit 2, a background image data storage unit 82 that stores data corresponding to an image (background image) that is a background of the composite image, and And a program storage unit 83 that stores various programs executed by the imaging apparatus 1. The background image data storage unit 82 also stores image data corresponding to an image (template image) prepared in advance as a template. The storage unit 8 is realized using a semiconductor memory such as a flash memory or a RAM (Random Access Memory) that is fixedly provided inside the imaging apparatus 1. The storage unit 8 may be provided with a function as a recording medium interface that reads information stored in the storage medium while storing information in a storage medium such as a memory card mounted from the outside.

  The control unit 9 is realized by a CPU (Central Processing Unit) or the like. The control unit 9 reads various programs from the program storage unit 83 of the storage unit 8 in response to an operation signal from the operation input unit 4, transmits a control signal to each unit constituting the imaging apparatus 1, and transfers data. To control the operation of the imaging apparatus 1. The control unit 9 includes an image processing unit 91, a display image generation unit 92, a composite image generation unit 93, a position adjustment unit 94, and a display control unit 95.

  The image processing unit 91 performs various types of image processing on the left-eye image data and right-eye image data output from the signal processing units 21 h and 22 h, and outputs them to the captured image data storage unit 81 of the storage unit 8. Specifically, the image processing unit 91 performs processing such as edge enhancement, color correction, and γ correction on the left eye image data and right eye image data output from the signal processing units 21h and 22h, respectively.

  The display image generation unit 92 is based on the image data generated by the image processing unit 91, the composite image data generated by the composite image generation unit 93 to be described later, or the image data stored in the captured image data storage unit 81. The image data of the display image displayed on the display unit 6 is generated. Specifically, the display image generation unit 92 generates 3D images or image data corresponding to 2D images as display images. Among these, when generating the display image data of the 3D image based on the image data generated by the image processing unit 91, the left-eye image and the right-eye image subjected to the image processing are set to a predetermined aspect ratio (for example, 3: 4). A 3D image is generated by cutting out each of the images. Note that the aspect ratio that the display image generation unit 92 cuts out from the left-eye image and the right-eye image may be changed by the changeover switch 43. Further, when generating the display image data of the 2D image, the display image generation unit 92 generates superimposed image data for displaying the left eye image and the right eye image in a superimposed manner, or generates the left eye image and the right eye image. Image data of a display image is generated using one of the images.

  The composite image generation unit 93 extracts a main subject image portion such as a person from the left eye image and the right eye image subjected to image processing by the image processing unit 91, and pastes them on a template image desired by the user, thereby combining the left eye composite image and the right eye composite. Generate an image. Details of this composite image generation processing will be described later.

  The position adjustment unit 94 is included in the composite image by changing the position of the main subject image in the left-eye and right-eye composite images generated by the composite image generation unit 93 according to the trajectory of the object that has touched the touch panel 7 from the outside. Adjust the parallax of the main subject image or move the position.

  The display control unit 95 controls the display unit 6 to display a 3D image or a 2D image corresponding to the image data generated as the display image by the display image generation unit 92 according to the setting by operating the 2D / 3D changeover switch 45. I do. Specifically, when displaying the 3D image on the display unit 6, the display control unit 95 displays the left-eye image and the right-eye image of the 3D image generated by the display image generation unit 92 in the horizontal direction on the display screen of the display unit 6. Control is performed to output 3D images alternately arranged for each pixel to the display unit 6.

  On the other hand, when displaying a 2D image on the display unit 6, the display control unit 95 changes the power applied to the parallax barrier 63 from the on state to the off state, thereby changing the parallax barrier 63 from the light shielding state to the transmissive state. Then, control is performed to cause the display unit 6 to output a left-eye image or a right-eye image as a display image generated by the display-image generating unit 92 or a superimposed image of the left-eye image and the right-eye image.

  In the imaging device 1 having the above-described configuration, a situation when the imaging unit 2 generates two images in which one end in the left-right direction of each field of view overlaps will be described. FIG. 3 is a schematic diagram illustrating a situation when the imaging unit 2 generates two images in which one end in the horizontal direction of each field of view overlaps. As shown in FIG. 3, first, the imaging unit 2 is arranged in parallel with a main subject OB1 (distance d1) and a background subject OB2 (distance d2) having different distances from the imaging unit 2 by a distance B1. By photographing with the first imaging unit 21 and the second imaging unit 22, right-eye image data and left-eye image data are generated.

  Thereafter, the display image generation unit 92 cuts out images corresponding to the right-eye image data and the left-eye image data generated by the first imaging unit 21 and the second imaging unit 22 with a predetermined aspect ratio, respectively, so that the right-eye image WR1 and A left-eye image WL1 is generated. FIG. 4 is a diagram illustrating an example of two images corresponding to the two image data generated by the imaging unit 2 under the situation illustrated in FIG. 3. In FIG. 4, the right eye image WR <b> 1 is generated when the display image generation unit 92 cuts out from an image corresponding to the right eye image data generated by the first imaging unit 21. In FIG. 4, the left-eye image WL <b> 1 is generated when the display image generation unit 92 cuts out from the image corresponding to the left-eye image data generated by the second imaging unit 22. FIG. 5 is a diagram illustrating an example of an image obtained by virtually superimposing the right eye image and the left eye image generated by the display image generation unit 92 under the situation illustrated in FIG. 3.

  4 and 5 indicate image areas corresponding to the image data generated by the first imaging unit 21 and the second imaging unit 22, respectively. Hereinafter, also in FIGS. 5 to 8, the broken line and the alternate long and short dash line mean the same image area as in FIG. 4 and the like.

  Next, a composite image generation process performed by the composite image generation unit 93 will be described. As shown in FIG. 3, the composite image generation unit 93 includes an image extraction unit 93a, a synthesis unit 93b, and a left and right image determination unit 93c.

  The image extraction unit 93a is configured to select a right-eye subject image A1 (R) and a left-eye subject representing the main subject OB1 from the right-eye image WR1 and the left-eye image WL1 (see FIG. 4) captured by the first imaging unit 21 and the second imaging unit 22. Images A1 (L) (these are collectively referred to as main subject image A1) are extracted. Specific extraction methods include, for example, the following three methods.

  FIG. 6 is a diagram illustrating a first example of a method for extracting a main subject image. As shown in FIG. 6, first, the image extraction unit 93a superimposes the right eye image WR1 and the left eye image WL1. At this time, as shown in FIG. 3, the parallax of the image increases between the right eye image WR1 and the left eye image WL1 as the subject (for example, the main subject OB1) exists in front of or in the center of the field of view. The parallax of the image becomes smaller as the subject (for example, the background subject OB2) exists in the peripheral portion. Therefore, the image extracting unit 93a is a region where the images do not match on the superimposed image of the right eye image WR1 and the left eye image WL1, that is, one end in the left-right direction of the main subject images A1 (R) and A1 (L). Extract the overlapping area (shaded white area). Furthermore, the right eye subject image A1 (R) and the left eye subject image A1 (L) can be extracted by extracting the extracted region and the region where the original right eye image WR1 and the left eye image WL1 overlap each other.

  FIG. 7 is a diagram illustrating a second example of a method for extracting the main subject image A1. As shown in FIG. 7, the right eye image WR1 and the left eye image WL1 are overlaid so that the images near the center of the screen match, and the matching area between the right eye image WR1 and the left eye image WL1 is extracted as the main subject image A1. You may do it. Alternatively, the two images may be overlapped so that the region where the sharpness is maximum (that is, the region in focus) overlaps in each of the right eye image WR1 and the left eye image WL1.

  FIG. 8 is a diagram illustrating a third example of a method for extracting the main subject image A1. As shown in FIG. 8, the areas where the light emitting unit 23 (FIG. 1) emits light and the strong light is reflected are the right eye subject image A1 (R) and the left eye subject image A1 (L), and the right eye image WR1 and the left eye image WL1. May be extracted from each.

  The composition unit 93b generates a composite image by pasting the main subject image A1 extracted by the image extraction unit 93a on a background image desired by the user. In the first embodiment, a template image selected by a user from among template images prepared in advance is used as the background image. FIG. 9 is a diagram showing an example of such a template image. Here, as the background image, a 2D image as shown in the template image TP1 of FIG. 9 may be used, or an image capable of 3D display may be used. When a 2D image is selected as the background image, the synthesizer 93b adds the right eye subject image A1 (R) and the left eye subject image A1 (L) to the same background image (template image TP1) as shown in FIG. ) Are respectively pasted to generate a right eye composite image WR2 and a left eye composite image WL2. The arrangement of the right-eye subject image A1 (R) and the left-eye subject image A1 (L) is as shown in FIG. 10B when the right-eye composite image WR2 and the left-eye composite image WL2 are virtually superimposed. Are determined so that one end in the left-right direction overlaps (that is, parallax occurs). When 3D display is performed based on such a right-eye composite image WR2 and left-eye composite image WL2, a 3D composite image WT2 in which the main subject image A1 is virtually raised on the template image TP1 is obtained (see FIG. 10C). .

  On the other hand, when the 3D image is selected as the background image, the combining unit 93b pastes the right eye subject image A1 (R) to the right eye background image and pastes the left eye subject image A1 (L) to the left eye background image. A right eye composite image and a left eye composite image are generated.

  The left and right image discriminating unit 93c is a right-eye image or a left-eye image in which an image composition material (a main subject image extracted by the image extraction unit 93a or a 3D image used as a background image) is an image for the right eye. Determine if it exists. For example, when the main subject image A1 in which the left and right ends overlap each other as shown in FIG. 5 is acquired, the left and right image determination unit 93c causes the main subject image A1 to protrude from the screen when displayed as a 3D image. As can be seen, the left-hand subject image is the left-hand subject image and the right-hand subject image is the left-eye subject image.

  Next, processing performed by the position adjustment unit 94 will be described. The position adjustment unit 94 responds to the trajectory of the object in contact with the touch panel 7 while the superimposed image WD2 of the right-eye composite image WR2 and the left-eye composite image WL2 shown in FIG. The parallax is adjusted by changing the relative positions of the right-eye subject image A1 (R) and the left-eye subject image A1 (L). Further, the position adjustment unit 94 changes the position with respect to the background image by moving the right-eye subject image A1 (R) and the left-eye subject image A1 (L) while maintaining the relative positional relationship.

  For example, as illustrated in FIG. 11A, when the user performs an input operation of touching the left-eye subject image A1 (L) and moving it to the right, the position adjustment unit 94 performs the right-eye according to the touch trajectory. The overlap between the subject image A1 (R) and the left-eye subject image A1 (L) is reduced (that is, the parallax is increased). As a result, as shown in FIG. 11B, in the 3D composite image WT2, a distance (hereinafter referred to as “jump distance”) D in which the main subject image A1 virtually jumps out in a direction orthogonal to the display screen is increased. On the other hand, when the user touches the left eye subject image A1 (L) and performs an input operation of moving leftward within a range not exceeding the right eye subject image A1 (R), the position adjustment unit 94 follows the touch trajectory. Accordingly, the overlap between the right eye subject image A1 (R) and the left eye subject image A1 (L) is increased (that is, the parallax is reduced). In this case, the pop-out distance D of the main subject image A1 is small in the composite image displayed as a 3D image.

  While performing such an operation, the user can check the pop-up distance D in real time by operating the 2D / 3D change-over switch 45 to switch to 3D display at any time. For example, as shown in FIG. 11, when the main subject image A1 is pasted so as to overlap an object (such as a character) in the template image TP1, the main subject image A1 may be projected beyond the character. The user may adjust the pop-up distance D so that the entire composite image looks natural.

  Also, as shown in FIG. 12, when the user touches both the right-eye subject image A1 (R) and the left-eye subject image A1 (L) and performs an input operation for moving in either the up / down / left / right direction, The adjustment unit 94 moves the positions of the right-eye subject image A1 (R) and the left-eye subject image A1 (L) with respect to the background image TP1 according to the touch trajectory without changing the relative positional relationship between them. . At that time, when the user performs an input operation to rotate the locus, the position adjustment unit 94 converts the right eye subject image A1 (R) and the left eye subject image A1 (L) in the plane of the background image TP1 according to the locus. You may rotate with.

  Further, as shown in FIG. 13A, when the user touches the left eye subject image A1 (L) and moves it to the left side of the right eye subject image A1 (R), the left eye subject image A1 (L) When an input operation is performed to reverse the left-right relationship with the right-eye subject image A1 (R) (that is, when the left end of the right-eye subject image A1 (R) and the right end of the left-eye subject image A1 (L) overlap) The position adjustment unit 94 adjusts the parallax so that the main subject image A1 appears to be virtually retracted by the retract distance D ′ in a direction orthogonal to the display screen (see FIG. 13B). For example, when it is desired to make the main subject image A1 appear behind the object (character or the like) in the template image TP1, the main subject image A1 may be shown retracted from the character. At this time, the user may adjust the retraction distance D ′ so that the entire synthesized image looks natural.

  Note that the position adjustment unit 94 may fix the parallax at the limit value when the touch trajectory exceeds the limit value of the parallax that can display the 3D image when adjusting the parallax of the main subject image A1.

  Next, processing performed by the imaging apparatus 1 will be described. FIG. 14 is a flowchart showing the operation of the imaging apparatus 1. In FIG. 14, first, the control unit 9 determines whether or not the power of the imaging device 1 is turned on (step S <b> 101). When the power supply of the imaging device 1 is on (step S101: Yes), the imaging device 1 proceeds to step S102. On the other hand, when the power of the imaging apparatus 1 is not turned on (step S101: No), the imaging apparatus 1 ends this process.

  Subsequently, the control unit 9 determines whether or not the imaging apparatus 1 is set to the shooting mode (step S102). When the imaging device 1 is set to the shooting mode (step S102: Yes), the display control unit 95 displays through images corresponding to the image data that the imaging unit 2 continuously generates at a constant minute time interval. According to the setting by operating the 2D / 3D selector switch 45, the display unit 6 displays the image in 2D or 3D (step S103).

  In step S104, the control unit 9 determines whether or not the imaging device 1 is set to the image composition mode. When the imaging device 1 is set to the image composition mode (step S104: Yes), the control unit 9 subsequently determines whether or not a template image is selected (step S105). If a template image has already been selected (step S105: Yes), the display control unit 95 displays the selected template image on the display unit 6 (step S107). On the other hand, if a template image has not yet been selected (step S105: No), the display control unit 95 displays a template image selection screen on the display unit 6 and allows the user to select a desired template image (step S106).

  FIG. 15 is a schematic diagram illustrating an example of a template image selection screen displayed on the display unit 6 in step S106. The template image selection screen 100 includes reduced images 101 to 106 of templates displayed as thumbnails based on template image data stored in the background image data storage unit 82, and an OK button 107. When a signal for selecting one of the reduced images and the OK button 107 is input due to an external object touching the touch panel 7 (step S106), a template image corresponding to the selected reduced image is displayed on the display unit 6. (Step S107).

  Note that the display control unit 95 displays the template image selection screen 100 on the display unit 6 and highlights the selected template image even when the template image has already been selected in step S105. You may make it make a user confirm.

  In step S108, the image extraction unit 93a extracts a main subject image A1 representing the main subject OB1 that has entered the field of view of the imaging device 1 from an image displayed as a through image. As a method of extracting the main subject image A1, any method described with reference to FIGS. 6 to 8 may be used.

  In subsequent step S109, the composition unit 93b generates a right-eye composite image and a left-eye composite image in which the main subject image A1 is pasted on the template image TP1, and the display control unit 95 displays these composite images on the display unit 6. (See FIG. 16). During this time, the user operates the 2D / 3D selector switch 45 to display a 2D image obtained by superimposing the right eye composite image and the left eye composite image, and a 3D image display based on the right eye composite image and the left eye composite image. Can be switched at a desired timing. FIG. 16 shows a case where the superimposed image WD2 is displayed. As shown in FIG. 16, the user can adjust the position of the main subject image A1 on the template image TP1 by changing the orientation of the imaging unit 2 with respect to the main subject OB1.

  In step S110, the position adjustment unit 94 determines whether or not a touch operation has been performed on the display area of the main subject image A1 on the superimposed image WD2 while the display unit 6 displays a 2D image. When there is a touch operation (step S110: Yes), the position adjustment unit 94 adjusts the parallax and the position of the main subject image A1 according to the touch trajectory (step S111). On the other hand, when there is no touch operation (step S110: No), the operation proceeds to step S112 while maintaining the original parallax.

  In step S112, the control unit 9 determines whether or not the release switch 42 is operated by the user and a release signal instructing photographing is input. If a release signal instructing shooting is input within a predetermined time after the start of live view display on the display unit 6 or after the image composition mode is set (step S112: Yes), the imaging unit 2 performs shooting (step S112). S113). At this time, the display control unit 95 may control the display unit 6 so as to perform a so-called REC view display for displaying a photographed image for a predetermined time.

  Subsequently, the control unit 9 stores the image data obtained by photographing in the photographed image data storage unit 81 and, as related information, an area extracted from the photographed image as the main subject image A1 (extraction area information) is selected. Information such as information specifying the template image TP1 (background image information), the position of the main subject image A1 on the template image TP1 (composite position information), and the amount of parallax adjusted (parallax information) are associated with the captured image data. (Step S114). The control unit 9 may separately generate main subject image data separately from the captured image data, and store background image information, composite position information, and parallax information in association therewith. Thereafter, the operation of the imaging apparatus 1 returns to step S101.

  On the other hand, when the release signal instructing photographing is not input within the predetermined time (step S112: No), the operation of the imaging device 1 returns to step S101.

  Next, a case where the imaging device 1 is not set to the image composition mode (step S104: No) will be described. In this case, the control unit 9 determines whether or not the release switch 42 is operated by the user and a release signal instructing photographing is input (step S115). When a release signal instructing shooting is input within a predetermined time after the start of the through image on the display unit 6 (step S115: Yes), the imaging unit 2 performs shooting (step S116), and the control unit 9 The photographed image data obtained by photographing is stored in the storage unit 8 (step S117). At this time, the display control unit 95 may control the display unit 6 to perform REC view display. Thereafter, the operation returns to step S101. On the other hand, after the start of the live view display, when the release signal for instructing photographing is not input within a predetermined time (step S115: No), the operation of the imaging device 1 returns to step S101.

  Next, the case where the imaging apparatus 1 is not set to the shooting mode in step S102 will be described (step S102: No). In this case, the imaging apparatus 1 executes a reproduction display process for displaying the captured image data on the display unit 6 (step S120).

  FIG. 17 is a flowchart illustrating the operation of the imaging apparatus 1 in the reproduction display process. In FIG. 17, first, the display control unit 95 causes the display unit 6 to display an image selection screen in which a plurality of images are displayed together based on the image file stored in the captured image data storage unit 81 (step S121).

  In step S <b> 122, the control unit 9 determines whether an image is selected from the image selection screen by operating the touch panel 7 by the user. If no image is selected (step S122: No), the operation returns to step S121. On the other hand, when the image is selected (step S122: Yes), the control unit 9 determines whether the image is a composite image based on the related information of the selected image (step S123). . For example, when the background image information is included in the related information, the control unit 9 determines that the image is a composite image.

  If the selected image is not a composite image (step S123: No), the display control unit 95 reads the captured image data corresponding to the image from the storage unit 8, and causes the display unit 6 to display an enlarged image (step S124). .

  On the other hand, when the selected image is a composite image (step S123: Yes), the composition unit 93b reads the captured image data (or subject image data) from the storage unit 8 and also obtains template image data based on the related information. Based on the read-out parallax information and the synthesized position information, synthesized image data in which the main subject image is pasted on the template image is generated. Further, the display image generation unit 92 generates display image data based on the combined image data, and the display control unit 95 displays an enlarged image of the combined image on the display unit 6 based on the display image data. It is displayed (step S125).

  FIG. 18 is a schematic diagram illustrating a display example of an image displayed on the display unit 6 in step S125. The display screen of the display unit 6 is provided with an image list display area 111 in which a plurality of reduced images 112 and 113 for image selection are displayed, and an enlarged image display area 114 in which the selected reduced image 112 is enlarged. At this time, the user operates the 2D / 3D changeover switch 45 to display a 2D image obtained by superimposing the right eye composite image and the left eye composite image, and to display a 3D image based on the right eye composite image and the left eye composite image composite image. The display can be switched at a desired timing. FIG. 18 shows a superimposed image WD2 displayed as a 2D image.

  While the display unit 6 displays the superimposed image WD2, the control unit 9 touches the touch panel 7 with respect to the display area of the main subject image A1 on the superimposed image WD2 (that is, outside the touch panel 7). It is determined whether or not a signal corresponding to the contact position is input (step S126). When there is a touch operation (step S126: Yes), the control unit 9 adjusts the parallax and the position of the main subject image A1 according to the touched locus (step S127). The method for adjusting the parallax and the position is the same as that described with reference to FIGS.

  When adjusting the parallax and the position, the user may input an operation by touching a finger on one place on the display area of the main subject image A1, as shown in FIG. 19, for example. As shown, an operation input (pinch operation) for touching two places on the display area of the main subject image A1 to widen the gap between the fingers in the left-right direction and conversely narrow the gap between the fingers is performed. Also good. In the latter case, the position adjustment unit 94 accepts such an operation as an input of a signal that increases (or decreases) the parallax of the main subject image A1.

  On the other hand, when there is no touch operation (step S126: No), the position adjustment unit 94 does not adjust the parallax and the position with respect to the main subject image A1, and the operation proceeds to step S128.

  When there is an operation input for ending the enlarged display of the currently displayed image due to a signal input for selecting one of the other reduced images 113 from the image list display area 111 (step S128: Yes), the control unit 9 Causes the storage unit 8 to overwrite and store the parallax, the position-adjusted parallax information, and the combined position information as related information of the captured image data (step S129). At this time, the control unit 9 displays a message as to whether or not to save the parallax information and the combined position information on the display unit 6 and overwrites the parallax information and the combined position information based on the signal input by the user. Anyway. On the other hand, when there is no operation input for ending the enlarged display of the currently displayed image (step S128: No), the operation returns to step S126.

  When the reproduction mode of the imaging apparatus 1 is ended by an input signal via the changeover switch 43 (step S130: Yes), the operation returns to the main routine. On the other hand, when the reproduction mode is continued (step S130: No), the operation returns to step S121.

  As described above, according to the first embodiment, the image of the main subject is extracted from the photographed image and is combined with a template image prepared in advance as a background. Therefore, a composite image in which a subject such as a familiar person is copied is displayed. It can be generated easily. In addition, when generating or displaying a composite image, the parallax can be changed by a touch operation on the superimposed image of the right eye image and the left eye image, so that the user can adjust the stereoscopic effect of the main subject image with an intuitive operation. It becomes possible to do. As a result, it is possible to display a 3D image in which the main subject image has a three-dimensional effect (feeling of popping out and retracting) with respect to the background image and is less likely to be uncomfortable.

  In the first embodiment described above, the main subject image is extracted by capturing the captured image while the template image is displayed on the display unit 6, but the template image is selected after the main subject image is extracted. Or may be displayed.

  In the first embodiment, an image captured by the user in advance may be stored in the background image data storage unit 82 as a background image, and a composite image may be generated using the image.

(Embodiment 2)
Next, an imaging apparatus according to Embodiment 2 of the present invention will be described. The imaging apparatus according to the second embodiment is based on an image signal acquired by photographing a display screen of a 3D television or the like that displays a 3D image by a frame sequential method that switches between a right-eye image and a left-eye image at high speed. And generating a 3D background image.

  FIG. 21 is a block diagram illustrating a configuration of the imaging apparatus according to the second embodiment. As illustrated in FIG. 21, the imaging apparatus 201 includes a 3D background image acquisition unit 96 that captures background image data that can be displayed in 3D from captured image data obtained by capturing a display screen of a 3D television or the like. Other configurations of the imaging apparatus 201 are the same as those shown in FIG.

  First, a background image generation method according to the second embodiment will be described. FIG. 22 is a schematic diagram illustrating a state in which the imaging device 201 captures an image of the display device 202 which is a frame sequential 3D television. As shown in FIGS. 23A and 23B, the right eye image and the left eye image are alternately displayed on the display screen 203 of the display device 202 at a predetermined cycle. When high-speed continuous shooting is performed on such a display screen at an imaging timing as illustrated in FIG. 23C, the right-eye image and the left-eye image are reflected in the captured image at predetermined intervals.

  The 3D background image acquisition unit 96 is used as a background image by capturing captured image data generated by high-speed continuous shooting at a predetermined timing while the imaging device 1 is set in the television (TV) shooting mode. Image data corresponding to the right eye image and the left eye image is acquired.

  The left and right image determination unit 93c of the composite image generation unit 93 determines whether the background image captured by the 3D background image acquisition unit 96 is the left or right background image. Here, as shown in FIG. 24, it is preferable that the image used as the background of the composite image be seen to be virtually retracted (away from the user 204) from the actual display screen P1. For this purpose, an image displayed on the right side of the screen at the position of the display screen P1 may enter the right eye of the user 204, and an image displayed on the left side of the screen may enter the left eye of the user 204. Therefore, the left and right image discriminating unit 93c sorts the image that is on the right side of the screen as the right eye background image and the image that is on the left side of the screen as the left eye background image (see FIG. 23D).

  Further, the control unit 9 extracts images with high sharpness from the images sorted as the right eye background image and the left eye background image, and stores them in the background image data storage unit 82.

  FIG. 25 is a flowchart showing the operation of the imaging apparatus 201. In FIG. 25, first, the control unit 9 determines whether or not the power of the imaging device 201 is turned on (step S201). When the power of the imaging device 201 is on (step S201: Yes), the imaging device 201 proceeds to step S202. On the other hand, when the power of the imaging apparatus 201 is not turned on (step S201: No), the imaging apparatus 201 ends the operation.

  In step S202, the control unit 9 determines whether or not the imaging apparatus 201 is set to the shooting mode. If the photographing apparatus 201 is not set to the photographing mode (step S202: No), the operation proceeds to step S120. The operation in step S120 is the same as that described in the first embodiment.

  On the other hand, when the imaging apparatus 201 is set to the imaging mode (step S202: Yes), the display control unit 95 displays the through image corresponding to the image data that the imaging unit 2 continuously generates at a constant minute time interval. Are displayed on the display unit 6 as a 2D image or a 3D image according to the setting by operating the 2D / 3D changeover switch 45 (step S203).

  In subsequent step S204, the control unit 9 determines whether or not the imaging device 201 is set to the television (TV) shooting mode (step S204). When the television shooting mode is not set (step S204: No), the operation directly proceeds to step S209.

  When the imaging device 201 is set to the television shooting mode (step S204: Yes), the control unit 9 determines whether or not a release signal instructing shooting is input by the user operating the release switch 42 (step S204: Yes). Step S205). When a release signal instructing shooting is input within a predetermined time after the start of live view display on the display unit 6 or after the image composition mode is set (step S205: Yes), the imaging unit 2 performs high-speed continuous shooting. Then, the 3D background image acquisition unit 96 captures the captured image data at a predetermined timing (step S206). Subsequently, the left and right image determination unit 93c performs a process of distributing an image corresponding to the captured image data captured by the 3D background image acquisition unit 96 into a left eye background image and a right eye background image (step S207). Further, the control unit 9 extracts images with high sharpness from the images assigned to the left eye background image and the right eye background image, and stores them in the background image data storage unit 82 (step S208).

  In subsequent step S209, the control unit 9 determines whether or not the imaging apparatus 201 is set to the image composition mode. When the image composition mode is set (step S209: Yes), the control unit 9 determines whether or not a background image used for image composition is set (step S210). For example, when a background image is generated in steps S204 to S208, it is determined that the generated background image is set.

  When the background image is set (step S210: Yes), the display control unit 95 displays the set background image on the display unit 6 (step S212). On the other hand, when the background image has not been set yet (step S210: No), the display control unit 95 displays the background image selection screen on the display unit 6 and causes the user to select a desired background image (step S211). On this background image selection screen, a background image generated in the past or a template image used in the first embodiment may be displayed.

  In step S213, the image extraction unit 93a extracts the main subject image from the image displayed as a through image. As a method for extracting the main subject image, any method described with reference to FIGS. 6 to 8 may be used.

  In step S214, the composition unit 93b generates a right eye composite image and a left eye composite image in which the main subject image is pasted on the background image. Further, the display image generation unit 92 generates display image data based on these composite images, and the display control unit 95 causes the display unit 6 to display the composite image based on the display image data. At this time, the display control unit 95 displays a 2D image obtained by superimposing the right-eye synthesized image and the left-eye synthesized image, or based on the right-eye synthesized image and the left-eye synthesized image, according to the setting by operating the 2D / 3D changeover switch 45. The display unit 6 is controlled to display a 3D image (see FIGS. 10B and 10C).

  In step S215, the position adjustment unit 94 determines whether a touch operation has been performed on the display area of the main subject image on the superimposed image while the display unit 6 displays the 2D image. If there is a touch operation (step S215: Yes), the position adjustment unit 94 adjusts the parallax of the main subject image A1 according to the touched locus (step S216). On the other hand, when there is no touch operation (step S215: No), the operation proceeds to step S217 with the original parallax.

  In step S217, the control unit 9 determines whether or not a release signal for instructing photographing is input by operating the release switch 42 by the user. When a release signal instructing shooting is input within a predetermined time after the start of live view display on the display unit 6 or after setting of the image composition mode (step S217: Yes), the imaging unit 2 performs shooting (step S217). S218). At this time, the display control unit 95 may control the display unit 6 to perform REC view display.

  Subsequently, the control unit 9 stores the image data obtained by photographing in the photographed image data storage unit 81, information for specifying the background image (background image information), and the position of the main subject image on the background image (composite). Position information) and related information including the amount of parallax adjusted in step S216 (parallax information) are stored in association with the captured image data (step S219). Thereafter, the operation returns to step S201.

  On the other hand, when the release signal instructing photographing is not input within the predetermined time (step S217: No), the operation directly returns to step S201.

  Next, a case where the imaging apparatus 201 is not set to the image composition mode (step S209: No) will be described. In this case, the control unit 9 determines whether or not a release signal instructing photographing is input (step S231). When a release signal instructing shooting is input within a predetermined time after the display unit 6 starts displaying a through image (step S231: Yes), the imaging unit 2 performs shooting (step S232), and the control unit 9 Photographed image data obtained by photographing is stored in the storage unit 8 (step S233). At this time, the display control unit 95 may control the display unit 6 to perform REC view display. Thereafter, the operation returns to step S201. On the other hand, if a release signal for instructing photographing is not input within a predetermined time after the through image display is started (step S231: No), the operation returns to step S201.

  According to the second embodiment described above, a 3D background image can be easily generated using a television capable of displaying a 3D image. Therefore, the user can synthesize and view a main subject image such as a familiar person on a background image on which a favorite character or landscape is projected. Further, according to the second embodiment, it is possible to generate and display a composite image without a sense of incongruity by adjusting the stereoscopic effect of the main subject image in accordance with the stereoscopic effect of the background image.

(Embodiment 3)
Next, an image communication system according to Embodiment 3 of the present invention will be described. Embodiment 3 is characterized in that a composite image generated by an imaging device is output and displayed on a display device such as a television capable of displaying a 3D image.

  FIG. 26 is a schematic diagram illustrating a configuration of an image communication system according to the third embodiment. An image communication system 300 illustrated in FIG. 26 is electrically connected to the imaging device 301 that is a digital stereo camera, and can receive image data from the imaging device 301 and display a 3D image. And a display device 302 as an external device.

  FIG. 27 is a block diagram illustrating a configuration of the imaging device 301. As illustrated in FIG. 27, the imaging device 301 includes a communication unit 10 that transmits and receives data to and from an external device, a transmission image generation unit 97, and a communication state detection unit 98. About another structure, it is the same as that of the imaging device 1 shown in FIG. Alternatively, the imaging apparatus 301 may further include a 3D background image acquisition unit 96 as in the imaging apparatus 201 illustrated in FIG.

  The transmission image generation unit 97 generates image data of a transmission image to be transmitted to the display device 302 for display on the display screen 303 of the display device 302 externally connected via the communication unit 10.

  The communication state detection unit 98 detects the communication state between the imaging device 301 and the display device 302. For example, when the imaging device 301 and the display device 302 are connected via HDMI (registered trademark), the communication state detection unit 98 detects the communication state between the imaging device 301 and the display device 302 according to the CEC protocol. To do. When the imaging device 301 and the display device 302 are connected via the Internet, the communication state detection unit 98 detects the communication state between the imaging device 301 and the display device 302 according to the TCP / IP protocol. Note that the communication between the imaging device 301 and the display device 302 may be wireless communication.

  For example, the display device 302 can perform 3D by a frame sequential method in which a right-eye image and a left-eye image displayed on the display screen 303 are switched at high speed, or a polarizing film method in which a polarizing film is incorporated in the display screen 303 formed of a liquid crystal panel. It is a 3D television that displays images. When a user views a 3D image displayed on such a 3D television, it is necessary to wear dedicated glasses 304 corresponding to the method. The glasses 304 applied in the case of the frame sequential method are liquid crystal shutter glasses having a configuration capable of closing an eye opposite to an eye for viewing a displayed image in synchronization with switching of images. In addition, the glasses 304 applied in the case of the polarization method have a configuration in which a polarizing filter is attached so that only the left eye image reaches the left eye and only the right eye image reaches the right eye.

  In such an image communication system 300, the imaging device 301 captures a subject and generates a composite image in the same manner as in the first embodiment while the photographing mode is set by operating the changeover switch 43 (FIG. 14). Alternatively, the imaging device 301 may generate a background image using the display device 302 as in the second embodiment (see FIG. 25). In addition, the imaging device 301 displays a captured image or a composite image on the display unit 6 provided in the imaging device 301 while the reproduction display mode is set by operating the changeover switch 43 (see FIG. 17).

  In addition, the imaging device 301 causes the display device 302 to display an image corresponding to the image displayed on the display unit 6 while being connected to the display device 302. Specifically, when the display image generating unit 92 generates display image data based on the captured image data or the composite image data, the transmission image generating unit 97 is based on the same captured image data or the composite image data. Image data for transmission is generated and transmitted to the display device 302 via the communication unit 10. At this time, whether or not to transmit the image data for transmission may be switched by operating the changeover switch 43. In addition, regardless of the display mode of the image in the imaging device 301, the display device 302 displays a superimposed image of the right eye image and the left eye image as a 2D image, and displays the right eye image or the left eye image as a normal 2D image. The mode to be displayed and the mode to display the 3D image based on the right-eye image and the left-eye image may be switched by operating the changeover switch 43.

  According to the third embodiment described above, the user can display and view the composite image generated in the imaging device on the external device (display device). Further, at that time, the user can adjust the parallax of the main subject image on the touch panel of the imaging device while displaying the composite image in 3D on the external device and checking the stereoscopic effect in real time. A synthetic image that looks natural on the display screen can be generated by an intuitive operation.

  In the first to third embodiments described above, the parallax information and the composite position information of the main subject image are stored as the related information of the image data. May be stored. As a result, it is possible to display a composite image in which a realistic stereoscopic effect is reproduced. In particular, in order to synthesize a three-dimensional image without a sense of incongruity in a three-dimensional background, repeated trial and error occurs by the user. . According to the first to third embodiments of the present invention, an intuitive operation is pursued, and a stereoscopic image can be confirmed on the spot while adjusting immediately (in real time). It has the effect of being easy to make.

  In the first to third embodiments described above, the parallax and the position of the main subject image on the composite image are adjusted by operating the touch panel, but in addition, a pointing device such as a mouse or a touch pen, Other various input devices may be used to adjust the parallax and the position by adopting the same concept described in the first to third embodiments.

  Furthermore, in the first to third embodiments described above, the imaging unit 2 is configured by two optical systems that respectively capture two images. However, images are continuously displayed at predetermined intervals while the imaging device moves. An image pickup unit having a function of picking up images, and from the images continuously picked up by the image pickup unit, the display image generation unit extracts two images in which one end in the left-right direction of the visual field overlaps each other, You may make it produce | generate the imaging | photography image for right eyes and left eyes.

  In the first to third embodiments described above, the case of generating a composite image that is a still image has been described. However, these embodiments may be applied to a moving image.

  The imaging apparatus according to the present invention can be applied to various electronic devices having a shooting function and a display function such as a digital video camera and a camera-equipped mobile phone in addition to the digital stereo camera.

  In the present invention, the display device as an external device is not limited to a 3D television, and any electronic device may be used as long as it has a display unit capable of displaying a 3D image.

1, 201, 301 Imaging device 2 Imaging unit 3 Attitude detection unit 4 Operation input unit 5 Clock 6 Display unit 7 Touch panel 8 Storage unit 9 Control unit 10 Communication unit 21, 22 Imaging unit 21a, 22a Lens unit 21b, 22b Lens drive unit 21c, 22c Aperture 21d, 22d Aperture drive unit 21e, 22e Shutter 21f, 22f Shutter drive unit 21g, 22g Image sensor 21h, 22h Signal processing unit 23 Light emitting unit 41 Power switch 42 Release switch 43 Changeover switch 44 Zoom switch 45 2D / 3D Switch 61 Backlight 62 Display panel 63 Parallax barrier 81 Captured image data storage unit 82 Background image data storage unit 83 Program storage unit 91 Image processing unit 92 Display image generation unit 93 Composite image generation unit 93a Image extraction unit 93b Synthesis unit 93 Left and right image discrimination unit 94 Position adjustment unit 95 Display control unit 96 3D background image acquisition unit 97 Transmission image generation unit 98 Communication state detection unit 100 Template image selection screen 101-106 Reduced image 107 OK button 111 Image list display area 112, 113 Reduced image 114 Enlarged image display area 202, 302 Display device 203, 303 Display screen 204 User 300 Image communication system

Claims (18)

An imaging unit that images a subject and generates image data;
A display unit capable of displaying a three-dimensional image based on the two images;
An image generation unit that generates two captured images corresponding to the right eye and the left eye, which are the basis of a three-dimensional image, based on the image data generated by the imaging unit;
An image extraction unit for acquiring two main subject images corresponding to the right eye and the left eye by extracting images representing the main subject excluding the background from the two captured images;
An image composition unit for generating two composite images corresponding to the right eye and the left eye by pasting the two main subject images extracted by the image extraction unit on a predetermined background image;
An input receiving unit that receives an input of an operation from the outside;
A position adjusting unit that adjusts a parallax by changing a relative position of the two main subject images in the two composite images according to an input received by the input receiving unit;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the main subject is an image located in front of or in the center of the field of view of the two captured images.   The imaging apparatus according to claim 2, wherein the main subject is an image having the largest parallax in the two captured images. A light emitting unit that illuminates the field of view of the imaging unit;
The imaging apparatus according to claim 2, wherein the main subject is an image of an area where reflected light is strongest when the light emitting unit emits light.   5. The image synthesis unit generates the two synthesized images in which a background is displayed as a two-dimensional image by pasting the two main subject images on the same background image. The imaging device according to any one of the above.   The image composition unit attaches the two main subject images to two background images in which one end in the left-right direction of the visual field overlaps each other, whereby the two composite images that can be displayed as a three-dimensional image. The imaging device according to any one of claims 1 to 4, wherein: An image acquisition unit for acquiring two display screen shot images corresponding to the right eye and the left eye from a shot image generated by high-speed continuous shooting of a display screen that displays a three-dimensional image in a frame sequential manner;
The image compositing unit generates the two composite images by pasting the two main subject images on the two display screen captured images acquired by the image acquisition unit, respectively. The imaging device according to any one of 1 to 4.   When the three-dimensional image based on the two display screen captured images acquired by the image acquisition unit is displayed, the two display screen captured images are displayed so that the three-dimensional image is virtually retracted from the display screen. The imaging apparatus according to claim 7, further comprising a left and right image discriminating unit that discriminates correspondence between a right eye and a left eye of the image.   A display that displays either a three-dimensional image based on two images or a two-dimensional image obtained by superimposing two images in accordance with a signal input from the outside on the display unit. The imaging apparatus according to claim 1, further comprising a control unit. The input receiving unit is a touch panel that is provided on the display screen of the display unit and receives an input of a signal corresponding to the position of an object that comes into contact from the outside.
The imaging apparatus according to claim 9, wherein the position adjustment unit adjusts a parallax of the main subject image according to a trajectory of an object touching the touch panel.   The position adjustment unit adjusts parallax in a two-dimensional image obtained by superimposing the two composite images by changing an overlapping region where one end in the left-right direction of the two main subject images overlaps according to the locus. The imaging apparatus according to claim 10.   The position adjustment unit further moves the two main subject images in accordance with the trajectory while maintaining a relative positional relationship in a two-dimensional image obtained by superimposing the two composite images. The imaging apparatus according to claim 10 or 11, wherein a position of the main subject image with respect to a background image is moved.   The position adjustment unit further rotates the two main subject images according to the trajectory while maintaining a relative positional relationship in a two-dimensional image obtained by superimposing the two composite images. The imaging apparatus according to claim 10, wherein the main subject image is rotated in a plane of a background image. A transmission image generation unit configured to generate a transmission image to be transmitted for display on the external device using the two images;
A communication unit that transmits the transmission image generated by the transmission image generation unit to the external device;
The imaging apparatus according to claim 1, further comprising:   The imaging apparatus according to claim 1, wherein the imaging unit includes two optical systems that respectively capture the two images. The imaging unit has a function of continuously capturing images at a predetermined interval while the imaging device moves,
The imaging apparatus according to claim 1, wherein the image generation unit extracts the two images from images continuously captured by the imaging unit. The display control unit
The imaging apparatus according to claim 9, wherein control is performed so that two image data corresponding to the two images are alternately arranged for each pixel in the horizontal direction on the display screen and output to the display unit. . The imaging device according to any one of claims 1 to 17,
A display device that is communicably connected to the imaging device according to a predetermined communication standard and displays a three-dimensional image based on the transmission image received from the imaging device;
An image communication system comprising:

Images