WO2012124331A1

WO2012124331A1 - Three-dimensional image pickup device

Info

Publication number: WO2012124331A1
Application number: PCT/JP2012/001798
Authority: WO
Inventors: 東郷　仁麿; 孝幸有馬; 井村　康治; 和之田中; 中村　剛; 郁雄渕上; 山口　徹
Original assignee: パナソニック株式会社
Priority date: 2011-03-17
Filing date: 2012-03-14
Publication date: 2012-09-20

Abstract

This three-dimensional image pickup device is provided with: two cameras having different optical axes, respectively; a distance measuring unit, which measures a distance to an object on the basis of parallax of two camera images picked up by the cameras; a first three-dimensional image generating unit, which generates a three-dimensional image on the basis of the two camera images; and a second three-dimensional image generating unit, which generates a three-dimensional image on the basis of the distance to the object and a camera image picked up by one of the two cameras. Corresponding to photographing states on the basis of the positions of the two cameras with respect to the object, switching is performed between three-dimensional image generation using the first three-dimensional image generating unit, and three-dimensional image generation using the second three-dimensional image generating unit.

Description

3D imaging device

The present invention relates to a camera-equipped portable electronic device such as a mobile phone having a 3D camera capable of 3D imaging and a digital still camera.

As a 3D (three-dimensional) shooting method, with two cameras (binary cameras) having an optical axis separated by about 6 cm to 8 cm, which is the distance between human eyes, the left image of the left viewing line and the right image of the right viewing line There is a way to shoot.

(First conventional example)
FIGS. 24 (a), (b) and (c) show a conventional structure example of a 3D camera consisting of two cameras. For example, details are described in Patent Document 1.

As shown in FIGS. 24A and 24B, the two

cameras

12 and 13 with an optical zoom are disposed apart from each other on the back surface of the display element 15 of the housing 11 of the imaging device.

FIG. 24C shows a block diagram of the imaging apparatus. The first camera 12 and the second camera 13 with optical zoom are controlled by the camera control means 18. At the time of zooming, the positions of the lenses 16 of the first camera 12 and the second camera 13 are adjusted based on the adjustment value of the look-up table so that the optical zoom magnifications of the first camera 12 and the second camera 13 coincide.

(Second conventional example)
Next, a second conventional example of the 3D camera will be described. For example, details are described in Patent Document 2. FIG. 25 shows a block diagram of the imaging apparatus.

Two

cameras

22 and 23 are disposed apart from each other on the back surface of the display element of the housing. The first camera 22 has an optical zoom, and the second camera 23 is a single focus camera without an optical zoom. Further, the number of pixels of the first camera 22 is larger than the number of pixels of the second camera 23.

The first camera 22 with an optical zoom and the second camera 23 of a single focus type are controlled by a camera control means 28. At the time of zooming, the optical zoom magnification of the first camera 22 and the electronic zoom magnification of the second camera 23 are adjusted so that the zoom magnifications of the first camera 22 and the second camera 23 match.

Since the resolution of the second camera 23 is lower than that of the first camera 22, resolution enhancement of the second camera image is performed as follows.

A region of the first camera image (referred to as a corresponding block) corresponding to a part of the second camera image (referred to as a block) is searched for by image processing, and a block of the second camera image is converted to a corresponding block of the first camera image replace.

As described above, the resolution of the second camera image can be increased, and the difference in resolution between the first camera image and the second camera image can be reduced.

(Third prior art example)
FIGS. 26 (a), (b) and (c) show an example of a conventional structure of a 3D camera consisting of two cameras. For example, details are described in Patent Document 3.

As shown in FIG. 26A, the two

cameras

32 and 33 are disposed apart from each other in the housing 31 of the imaging apparatus. As shown in FIG. 26 (b), it is possible to mechanically change the distance between optical axes (baseline length) of the camera and to change the angle of the optical axis of the camera.

FIG. 26C shows a block diagram. The first camera 32 and the second camera 33 are controlled by the camera control means 38. At the time of 3D imaging, position adjustment (parallax adjustment) of the first camera 32 and the second camera 33 in the horizontal direction is also performed.

Here, the parallax means the difference in the horizontal position of the left and right images with respect to a certain subject. By controlling the magnitude of the left and right parallax, the subject can be freely placed in front of or behind the 3D display screen.

FIG. 27 shows the relationship between the subject distance from the 3D camera to the subject and the parallax angle (convergence angle) of the left and right images when the optical axes of the 3D camera are parallel.

As the subject distance decreases, the parallax angle rapidly increases. Also, as the distance between the optical axes (baseline length) of the two cameras increases, the parallax angle increases, and the 3D effect (feeling of depth) increases, while the feeling of popping out on the 3D display screen becomes too strong to be three-dimensional It also has the task of

The three-dimensional image reproduced on the 3D display may not be in front of or behind the display screen. When the amount of deviation with respect to the convergence angle on the display screen becomes a certain value or more, comfortable stereoscopic vision becomes difficult. Furthermore, if this shift amount becomes large, stereoscopic vision itself can not be performed.

Generally, it is said that a comfortable stereoscopic view is possible if the parallax angle is a difference of about ± 1 degree or less with respect to the display screen.

For example, when the base length is 6.5 cm and the subject distance is from about 1 m or more to infinity, there is no problem because the parallax angle is 2 degrees or less (± 1 degree or less) as shown in FIG. When it comes to an object at close range less than 1 m, comfortable stereoscopic vision can not be achieved.

Therefore, in Patent Document 3, a method is compatible with the 3D effect (= depth feeling) and the viewability by changing the base length of the 3D camera and the optical axis angle of the left and right cameras, that is, the convergence angle according to the 3D photographed image. Have been described.

When short-distance and long-distance subjects are mixed, shortening the baseline length of the camera reduces the left-right parallax and provides an easy-to-see 3D image. However, the 3D effect tends to be smaller.

For example, when the base length is reduced from 6.5 cm to 3 cm, the parallax angle decreases as shown in FIG. 27. Therefore, the parallax angle can be made 2 degrees or less even for a short distance object of about 50 cm.

Further, while the optical axes of the two cameras are parallel, the convergence angle increases as the object approaches the camera, and the display screen jumps out, making it difficult to view stereoscopically.

Therefore, when the subject is close, as shown in FIG. 26 (b), the optical axes of the two cameras are diagonally crossed so that the parallax at the time of 3D display can be reduced, and 3D is easy to see near. An image is obtained.

As described above, since adding a function to mechanically change the convergence angle of two cameras causes a camera to be large and heavy, Patent Document 4 describes a method of electronically changing the convergence angle.

The convergence angle can be changed by adjusting the cutout horizontal position of the left and right images.

However, with this electronic method, although the parallax of some objects can be adjusted, it is not possible to simultaneously reduce the parallax of all objects from a short distance to a long distance.

Patent Document 5 describes a method of generating a parallax image such that a specific object in a subject (object) in the 3D expression space falls within the recommended parallax range.

For example, the parallax adjustment is performed so that the designated object is displayed near the surface of the display screen.

(Fourth conventional example)
FIGS. 28 (a) and 28 (b) show an example of the conventional structure of a 3D camera consisting of a twin-lens camera.

The first camera 42 and the second camera 43 are disposed apart from each other in the housing 41 of the imaging device. On the back surface of the housing 41, a display element 45 for displaying a 3D photographed image is disposed. The display element 45 is, for example, a parallax barrier type naked eye 3D liquid crystal or the like.

FIG. 29 shows a state where the display screen is vertically elongated by rotating the case 41 by 90 degrees. When the subject is vertically long, shooting is often performed in this manner.

At this time, since the first camera 42 and the second camera 43 are vertically disposed, parallax in the vertical direction occurs on the subject of the left and right images. For this reason, the conventional twin-lens 3D camera shown in FIGS. 28 (a) and 28 (b) can not perform 3D vertical shooting. In addition, 3D display of the image is also impossible.

Therefore, in the case of vertical shooting, a solution method based on physical change of the arrangement of twin-lens cameras is proposed, for example, in Patent Document 6.

The 3D camera structure of patent document 6 is shown to FIG. 30 (a), (b). FIG. 30 (a) shows landscape shooting, and FIG. 30 (b) shows portrait shooting.

In the case of the horizontal shooting, shooting is performed by two

cameras

52 and 53 which are disposed apart from each other in the direction of the long side of the display element 55 as in the conventional case. The camera sensors 57 of the left and right cameras are both arranged horizontally long.

On the other hand, at the time of vertical shooting, the arrangement of the camera sensor 57 is changed from horizontal to vertical by a mechanism in which the first camera 52 and the second camera 53 rotate 90 degrees with respect to the camera optical axis.

The display element 55 also rotates 90 degrees in the same direction as the rotation direction of the camera, thereby enabling display of a vertically long 3D image.

The display device 55 needs a 3D display device capable of vertical display and horizontal display.

For example, by switching the barrier direction between vertical and horizontal using naked-eye 3D liquid crystal having barrier liquid crystal, horizontal display and vertical display of a 3D image are possible.

Note that instead of rotating the display element 55 itself, only the display image may be rotated.

In addition to the first to fourth conventional examples described above, as described in, for example, Patent Document 7, parallax information (or distance information) is another method of generating a 3D image from a 3D camera. There is a method of using). After measuring the parallax from the main image and the sub image of the twin-lens camera including the main image sensor and the sub image sensor disposed spatially apart, it is possible to generate a 3D image from the parallax information and the main image.

Japanese Patent No. 3303254 Japanese Patent Application Laid-Open No. 2005-210217 Japanese Patent Application Laid-Open No. 07-167633 Japanese Patent Application Laid-Open No. 08-251625 Japanese Patent Application Laid-Open No. 2004-220127 Japanese Patent Application Laid-Open No. 10-224820 Japanese Patent Laid-Open Publication No. 2005-20606

The 3D optical zoom camera described in Patent Document 1 described in the first prior art needs two expensive and large optical zoom cameras, so there is a problem that the portable terminal becomes expensive and large. .

On the other hand, the 3D optical zoom camera described in Patent Document 2 described in the second prior art is inexpensive and can be miniaturized because one side is a single-focus one-way camera. It takes time for software processing of resolution conversion to take moving pictures. Further, there may be a case where corresponding blocks (or corresponding pixels) of the optical zoom camera image and the single focus camera image can not be found. At this time, the single focus camera image is an image partially deteriorated in low resolution.

In addition, since the processing time of 3D image generation with the 3D camera described in patent document 7 takes time for 3D image generation, moving image shooting is difficult.

Although the first and second conventional examples assume that the

first cameras

12 and 22 have an optical zoom function, they are simply a 3D camera consisting of a high resolution third camera and a low resolution fourth camera. In the case of the same problem occurs. The issues are described below.

An image cut out from the third camera with the maximum number of pixels of AL is taken as a third camera generated image, and the number of pixels is BL (BL ≦ AL). On the other hand, an image cut out of the fourth camera (AR <AL) with the maximum number of pixels of AR is taken as a fourth camera generated image, and the number of pixels is BR (BR ≦ AR). Here, for simplicity, it is assumed that the aspect ratio of the image is constant.

When the ratio (BR / BL) of the number of pixels BR to the number of pixels BL is reduced, the 3D image having the third camera generated image and the fourth camera generated image as the left and right images has a large resolution difference and becomes a difficult 3D image . As described above, when the resolutions of the left and right images are different, the image quality of the 3D image is degraded.

The 3D camera described in Patent Document 3 described in the third prior art can achieve both the 3D effect and the viewability by changing the relative arrangement of the cameras, but moves the two cameras continuously. Since the function is required, there is a problem that the camera becomes expensive and large.

On the other hand, since the 3D camera described in Patent Document 4 can change the horizontal distance (parallax) of the subject in the left and right images depending on the image cutout position, a complicated mechanical structure as in Patent Document 3 is unnecessary. Only the adjustment of parallax in the horizontal direction is possible, and it is not possible to adjust the height of the sense of depth between the subjects (3D sense). Therefore, in the 3D image in which the short distance subject and the long distance subject are shown, it has not been possible to adjust so that all subjects have a comfortable parallax.

In particular, when the distance of the short distance object is, for example, less than 1 m and very close, the feeling of popping up on the display screen is too strong to be seen. Therefore, if the parallax of the short distance object is reduced to weaken the feeling of popping out, a distant object may be placed in the back and it may be difficult to view or may not be stereoscopically viewable.

Further, according to the method described in Patent Document 5, the sense of depth and the parallax angle (convergence angle) can be adjusted according to the position of the object (object), but the problem is that soft processing time is required for 3D image generation. was there.

In the 3D imaging with the twin-lens camera described in the fourth conventional example, it was not possible to perform vertical imaging while rotating the casing so that the display screen is vertically long.

On the other hand, Patent Document 6 proposes a method capable of realizing 3D vertical shooting, but a mechanism in which two cameras physically rotate with respect to the camera optical axis to perform vertical 3D shooting, The rotation mechanism of a display element is required, and the subject that an imaging device enlarges occurred.

Thus, depending on the type or number of pixels of the binocular camera provided in the 3D imaging apparatus, the distance to the subject, or the imaging state of the binocular camera with respect to the subject, suitable methods for generating a 3D image are It is different. However, in the related art, a 3D imaging apparatus having a simple configuration capable of generating a 3D image by an appropriate method according to a shooting state has not been realized.

An object of the present invention is to provide a simple 3D imaging apparatus capable of generating a 3D image by a method suitable for an imaging state.

Next, means for solving the above problems will be described.

A 3D imaging apparatus according to the present invention includes: two cameras having different optical axes; a distance measuring unit that measures a distance from an object based on parallax of two camera images captured by each of the two cameras; A 3D image is generated based on a first 3D image generation unit that generates a 3D image from an image, a distance to an object measured by the distance measurement unit, and a camera image captured by one of the two cameras And generating a 3D image by the first 3D image generation unit and a 3D image by the second 3D image generation unit according to a photographing state based on the positions of the two cameras with respect to the subject. Switch to one of the generation of.

In the 3D imaging apparatus according to the present invention, the two cameras are a first camera with an optical zoom function and a single-focus second camera, and the zoom magnification by the first camera is smaller than a predetermined magnification. Sometimes, the first 3D image generation unit generates a 3D image from the optical zoom image of the first camera and the electronic zoom image of the second camera, and when the zoom magnification is larger than the predetermined magnification, the second 3D image is generated. The 3D image generation unit generates a 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by the first camera.

With this configuration, it is possible to generate a 3D image in a short time by generating a 3D image from the images of the first camera and the second camera at low magnification. Further, by generating a 3D image from an optical zoom image at high magnification, 3D imaging with high zoom magnification can be realized while maintaining high resolution with an inexpensive camera configuration.

In the 3D imaging apparatus of the present invention, the two cameras are a third camera and a fourth camera having a smaller number of pixels than the third camera, and the number of pixels of the 3D image to be generated is a predetermined number of pixels When smaller than the above, the first 3D image generation unit generates a 3D image from the camera image captured by the third camera and the camera image captured by the fourth camera, and the number of pixels of the 3D image generated is the above When the number of pixels is larger than a predetermined number of pixels, the second 3D image generation unit generates a 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by the third camera. .

With this configuration, it is possible to generate a 3D image in a short time by generating a 3D image from the images of the third camera and the fourth camera at low magnification. In addition, by generating a 3D image from the third camera image at high magnification, 3D imaging with a high zoom magnification can be realized while maintaining high resolution with an inexpensive camera configuration. In addition, the image quality deterioration of the 3D image having a large difference in resolution between the left and right images is suppressed.

Further, in the 3D imaging device of the present invention, the two cameras are a third camera and a fourth camera having a smaller number of pixels than the third camera, and in the case of 3D moving image shooting, the first 3D image generation The unit generates a 3D image from the camera image captured by the third camera and the camera image captured by the fourth camera, and in the case of 3D still image shooting, the second 3D image generator converts the distance measurement unit A 3D image is generated based on the measured distance to the subject and the camera image captured by the third camera.

With this configuration, it is possible to generate 3D moving images in a short time by generating 3D images from images of the third camera and the fourth camera at the time of moving image shooting. In addition, at the time of still image shooting, by generating a 3D image from the third camera image, it is possible to realize 3D still image shooting with a high zoom magnification while maintaining high resolution with an inexpensive camera configuration. In addition, the image quality deterioration of the 3D image having a large difference in resolution between the left and right images is suppressed.

Further, in the 3D imaging apparatus according to the present invention, in the long distance imaging mode in which the distance to the subject measured by the distance measuring unit is a predetermined value or more, the first 3D image generating unit captures one of the two cameras. The first 3D image is generated from the captured camera image and the camera image captured by the other of the two cameras, and in the short distance shooting mode, the distance to the subject measured by the distance measuring unit is less than the predetermined value. The second 3D image generation unit generates a second 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by one of the two cameras, and the second 3D image is generated. The parallax angle difference between the first object and the second object at different parallax angles in the image is smaller than the parallax angle difference between the first object and the second object in the first 3D image.

The 3D imaging device of the present invention further includes a display unit that outputs the first 3D image and / or the second 3D image.

With this configuration, by using the distance measuring unit in the short distance shooting mode, both long distance and short distance shooting can be performed with an inexpensive camera configuration, and 3D shooting that achieves both 3D effect and easy viewing can be realized. .

Further, in the 3D imaging device of the present invention, the distance measuring unit calculates the closest distance of the third object closest to the 3D imaging device, and when the closest distance is equal to or more than the predetermined value, the far distance When the shooting mode is selected and the closest distance is less than the predetermined value, the short distance shooting mode is selected.

With this configuration, it is not necessary for the user to switch between the short distance imaging mode and the long distance imaging mode by measuring the closest object distance by the distance measuring unit.

Further, the 3D imaging device of the present invention includes a housing and a rectangular display element disposed in the housing, and the two cameras are arranged side by side in the long side direction of the display element, In the vertical 3D imaging mode in which the long side of the display element is vertical, the distance measuring unit measures the distance from the parallax of the display element to the subject of the two camera images captured by each of the two cameras. The second 3D image generation unit generates a 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by one of the two cameras.

With this configuration, by generating a 3D image from the distance to the subject, both vertical 3D imaging and horizontal 3D imaging can be made compact and inexpensive without physically changing the arrangement of the cameras.

Further, in the 3D imaging apparatus of the present invention, in the horizontal 3D imaging mode in which the long side of the display element is horizontal, the first 3D image generation unit generates a 3D image from camera images captured by each of the two cameras. Generate

With this configuration, it is possible to shorten the generation time of the 3D image in the horizontal 3D imaging mode.

The 3D imaging apparatus according to the present invention further includes an orientation sensor that detects whether the long side of the display element is horizontal or vertical, and the vertical 3D imaging mode or the horizontal 3D imaging is performed according to the detection result of the orientation sensor. Switch to mode.

With this configuration, automatic switching to the vertical 3D imaging mode according to the output result of the posture sensor eliminates the need for switching by the user.

Further, in the 3D imaging apparatus according to the present invention, the display element has a 3D display function, displays a 3D preview in the horizontal 3D imaging mode, and either one of the two cameras in the vertical 3D imaging mode The preview display by the display element is performed with a 2D image using an image.

With this configuration, even in the vertical 3D imaging mode, preview display can be performed without depending on the generation time of the 3D image.

Further, in the 3D imaging apparatus according to the present invention, the display element is a pseudo 3D image in which parallax is added in a short side direction of the display element from one camera image of the two cameras in the vertical 3D imaging mode. The preview display is performed.

With this configuration, the preview in the vertical 3D shooting mode can be displayed in a pseudo 3D manner, and the difference from the 2D shooting can be easily recognized.

Further, in the 3D imaging device of the present invention, parallax settings of left and right images in the horizontal 3D imaging mode and the vertical 3D imaging mode can be set independently.

With this configuration, it is possible to set parallaxes that are respectively optimal for vertical shooting and horizontal shooting.

Further, in the 3D imaging device of the present invention, the parallax setting value of the vertical 3D shooting mode is for short distance rather than the horizontal 3D shooting mode.

This configuration makes it easy to perform short distance shooting during vertical shooting.

The 3D imaging apparatus according to the present invention includes a first camera, an optical axis different from the optical axis of the first camera, and a second camera having a larger angle of view than the first camera, and the first camera. A camera control unit for controlling the second camera to synchronously capture a camera image of a shooting range equal to the angle of view with a shooting time of the first camera; a camera image of the first camera and a camera of the second camera And 3D image generation unit for generating a 3D image from the image.

With this configuration, it is possible to electrically obtain left and right images for a 3D image without performing an operation for changing the angle of view of the camera. For this reason, it is possible to obtain 3D images of a moving subject.

In the 3D imaging apparatus according to the present invention, the camera control unit changes the frequency of the clock signal input to the second camera so that the imaging time of the camera image of the second camera is the imaging time of the first camera Synchronize to

With this configuration, it is possible to cope with the change of the angle of view without changing the setting of the blank period of the camera and the like.

According to the present invention, it is possible to provide a simple 3D imaging apparatus capable of generating a 3D image by a method suitable for an imaging state.

(A), (b), (c) Schematic structural view of the 3D imaging apparatus according to the first embodiment of the present invention The figure which shows the production | generation method of the 3D image at the time of the zoom of Embodiment 1 of this invention. Figure showing subjective evaluation results of asymmetric 3D images Flow chart of 3D imaging according to the first embodiment of the present invention Flow chart of second 3D imaging according to the first embodiment of the present invention (A), (b), (c) The figure which shows the production | generation method of 3D image of Embodiment 2 of this invention. Diagram showing the relationship between the number of pixels and the resolution ratio in the first 3D image generation method Flow chart of 3D imaging according to the second embodiment of the present invention (A), (b), (c) Schematic structural view of the 3D imaging apparatus according to the third embodiment of the present invention (A), (b) A diagram showing a general 3D image capturing method FIG. 10 is a diagram showing a 3D image generation method according to Embodiment 3 of the present invention (A), (b) The figure which shows the detail of the 3D image generation method of Embodiment 3 of this invention 3D imaging flowchart of the third embodiment of the present invention (A), (b), (c) Schematic structural view of the 3D imaging apparatus according to the fourth embodiment of the present invention (A), (b) Schematic of horizontal shooting mode and vertical shooting mode according to the fourth embodiment of the present invention The figure which shows the 3D image generation method of Embodiment 4 of this invention. 3D imaging flowchart of the fourth embodiment of the present invention 3D imaging flowchart of the fifth embodiment of the present invention Second 3D imaging flowchart of the fifth embodiment of the present invention The figure which shows the preview display image of the 2nd 3D imaging | photography flowchart of Embodiment 5 of this invention. (A), (b) The figure which shows the 3D image generation method of Embodiment 6 of this invention (A), (b), (c) Schematic structural view of a 3D imaging apparatus according to a seventh embodiment of the present invention The figure which shows the image which the sensor of each camera of Embodiment 7 of this invention acquired. (A), (b), (c) Schematic structure diagram of a conventional 3D imaging device Schematic structure diagram of a conventional 3D imaging device (A), (b), (c) Schematic structure diagram of a conventional 3D imaging device Diagram of graph showing relationship between subject distance and parallax angle (A), (b) Schematic structure diagram of a conventional 3D imaging device The figure which shows a mode when the housing | casing 41 is rotated 90 degree | times and a display screen is made longitudinally long. (A), (b) Schematic structure diagram of a conventional 3D imaging device

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1
FIG. 1A, FIG. 1B, and FIG. 1C are schematic structural views of a 3D imaging apparatus according to the present embodiment.

Hereinafter, in the first embodiment, the description will be made on the assumption of a portable terminal device having a single housing, but in addition, an electronic device having a small camera such as a foldable portable terminal or a digital still camera (DSC) is described. The same applies to the case of wearing.

As shown in FIGS. 1A and 1B, a 3D camera 104 including a first camera 102 and a second camera 103 is disposed on the back of the display element 105 of the rectangular casing 101 of the mobile terminal device. The first camera 102 is a camera with an optical zoom function, and the second camera 103 is a single focus camera without an optical zoom function. Since the second camera 103 is cheaper and smaller than the first camera 102, the portable terminal can be made smaller or thinner than when two cameras with an optical zoom in FIG. 24A are used. .

Although the first camera 102 with an optical zoom function is a camera for the left image in FIG. 1A, it may be a camera for the right image.

FIG. 1C shows a block diagram of the portable terminal device. The first camera 102 and the second camera 103 are controlled by the camera control means 108, and the photographed image is stored in the storage means 110 or displayed using the display element 105.

The first camera 102 comprises a sensor 107 and a lens 106 and is provided with an optical zoom function, and the position of the lens 106 is controlled by the zoom control signal 109 to obtain a desired zoom magnification.

FIG. 2 shows a 3D image generation method at the time of zooming.

First, a 3D image generation method when the zoom magnification N is smaller than the switching magnification N0 will be described. This is called a first 3D image generation method.

The first camera 102 (left camera) acquires an optical zoom image according to the zoom magnification N. Further, an image of the same angle of view as the optical zoom image is cut out from the image of the second camera 103 (right camera), and then an electronic zoom image in which the number of pixels is adjusted (= resized) to the same number of pixels as the optical zoom image is created. A 3D image is generated by using the optical zoom image [1] as a left image and the electronic zoom image [2] as a right image.

Note that since the resolution of the electronic zoom image is relatively reduced as the magnification N is larger than the optical zoom image, the first 3D image generation method is performed only when the zoom magnification N is smaller than N0.

Assuming that the number of vertical pixels of the original image indicating the resolution of the right and left images is YL and YR, and the number of horizontal pixels is XL and XR, YR (or XR) is larger than YL (or XL) when the zoom magnification is large. It becomes smaller.

When the resolutions of the 3D left and right images are different, the inventors conducted a subjective evaluation as to whether or not the 3D image looks comfortable as a result.

With regard to four types of 3D images, whether there is no sense of incongruity as a 3D image when the number of pixels of the right image is made smaller than that of full HD with respect to an image whose left image is full HD (pixel number 1920 × 1080 dots) It shows the result of evaluation.

The vertical pixel number ratio represents the ratio of the vertical pixel number YR of the right image to the vertical pixel YL (= full HD 1080 dots) of the left image, and means that the smaller the resolution difference between the left and right images, the smaller the ratio. .

According to the inventor's subjective evaluation results, it was found that if YR / YL is about 0.4 or more, the reduction in the resolution of the right image is not a concern, and an image with less discomfort can be obtained as a 3D image.

Therefore, assuming that a zoom magnification at which YR / YL is, for example, 0.4 or more is N0, it can be understood that a 3D image can be generated by the first 3D image generation method described above when N <N0. The

In the case of N> N0, the resolution of the right image is lowered, and a good 3D image can not be obtained.

Therefore, next, a 3D image generation method when the zoom magnification is larger than N0 will be described. This will be called a second 3D image generation method.

The second 3D image generation method is a method of calculating or estimating distance information (depth information) for each pixel or minute region, and generating a 3D image from the distance information.

As a method of calculating depth information, there is also a method of estimating from the brightness of a pixel, etc., but a more accurate measurement method uses parallax information (= distance information) for each pixel or minute area from left and right images of a twin-lens camera It is a method of obtaining and is described, for example, in Patent Document 7.

Here, a method of obtaining parallax information (= distance information) for each pixel or a minute area from the left and right images of a twin-lens camera is adopted.

From the left and right images [3] and [4] of FIG. 2, the parallax information (distance information) of each pixel is obtained by establishing correspondence with each pixel (or a minute region) using correlation calculation of image processing or the like. [5] can be calculated.

Left and right images [6] and [7] are obtained from the first camera left image [3] and disparity information [5].

Since the first camera image [3] with high resolution is used for 3D image generation, both left and right images can be made high resolution images.

Accordingly, when the magnification is high (N> N0), a high resolution 3D image can be obtained by using the second 3D image generation method.

FIG. 4 shows a 3D imaging flowchart at the time of zooming according to the first embodiment of the present invention.

As described above, by combining the first 3D image generation method and the second 3D image generation method, high resolution 3D imaging can be performed in a wide zoom magnification range.

Even in the case of low magnification (N <N0), it is possible to use the second 3D image generation method, but the second 3D image generation method takes processing time, and therefore the resolution reduction of the second camera image is acceptable. Therefore, the first 3D image generation method is more desirable than the second 3D image generation method.

Also, in the case of 3D moving image shooting, the second 3D image generating method takes processing time and the frame rate of the moving image becomes small, so the first 3D image generating method is more desirable than the second 3D image generating method.

Therefore, instead of switching the first 3D image generation method and the second 3D image generation method according to the zoom magnification, a method of switching between a moving image and a still image is also effective.

FIG. 5 shows a 3D imaging flowchart at this time. In the case of moving image shooting, a first 3D image generation method with high processing speed is adopted, and in the case of still image shooting, a high resolution second 3D image generation method is adopted.

Thus, a 3D image can be generated without reducing the frame rate of the 3D moving image.

Second Embodiment
In the second embodiment, a method of changing the 3D image generation method according to the number of pixels of the 3D image to be generated will be described.

In the second embodiment, a mobile terminal device having a single housing is described, but in addition, the mobile terminal device is attached to an electronic device having a small camera such as a foldable mobile terminal or a digital still camera (DSC). The same is true for the case.

FIG. 6A is a block diagram of the 3D imaging apparatus according to the second embodiment.

Here, although the third camera 111 and the fourth camera 112 both have no optical zoom function, they may have an optical zoom function.

The 3D image generation method shown in FIG. 6B is the same as the first 3D image generation method of the first embodiment.

Hereinafter, the number of pixels indicates the number of vertical pixels × the number of horizontal pixels, and the resolution means the number of pixels in the original image. Also, for simplicity, it is assumed that the aspect ratio of the image is constant.

An image generated from the third camera 111 with the maximum number of pixels of AL ([6]) is a third camera generated image ([8]), and the number of pixels is BL (BL ≦ AL). On the other hand, an image generated from the fourth camera 112 of which the maximum number of pixels is AR (where AR <AL) ([7]) is a fourth camera generated image ([9]), and the number of pixels is BR (BR ≦ AR) I assume.

[8] A resized image to an image of pixel number C is a [10] image, and a resized image of [9] image to an image of pixel number C is a [11] image, from [10] and [11] Get a 3D image.

FIG. 7 shows an example of a graph of resolution ratio P ((right resolution) / (left resolution)) with respect to the number C of pixels of the 3D image.

When the number of pixels C is smaller than BR, the resolution of [10] decreases from BL to C, and the resolution of [11] also decreases from BR to C. Therefore, the ratio of the left and right resolutions is (left resolution) / (right resolution) = C / C = 1.

On the other hand, when the number of pixels C is larger than BR and smaller than BL, the resolution (number of pixels) of [10] decreases from BL to C, and the resolution of [11] remains BR. Therefore, the ratio of the left and right resolutions is (right resolution) / (left resolution) = BR / C> BR / BL.

On the other hand, when the pixel number C is larger than BL, the resolution of [10] remains BL, and the resolution of [11] also remains BR. Therefore, the ratio P of the left and right resolutions is (left resolution) / (right resolution) = BL / BR.

Here, if a certain resolution at which the resolution ratio P is, for example, 0.3 × 0.3 = 0.09 or more is set as P0, and the resolution C of the 3D image at this time is C0, FIG. As understood from the result of the above, if C <C0, the resolution ratio P is larger than P0, so there is no problem in the 3D image using the first 3D image generating method.

On the other hand, when C> C0, the first 3D image generation method is not suitable because the resolution difference is large at the resolution ratio P0 or less. Therefore, when C> C0, a second 3D image generation method is used to generate a 3D image.

FIG. 6C shows a second 3D image generation method. Further, FIG. 8 shows a flowchart of 3D imaging according to the second embodiment of the present invention.

When C> C0, the distance information [12] is calculated from the third camera image [10] and the fourth camera image [11], and the third camera image [10] of pixel number C with high resolution and the distance information [10] Obtain a 3D image from 12]. The generation method is the same as the second 3D image generation method of the first embodiment. Thus, 3D images with the same resolution of the left and right images can be obtained.

As described above, by changing the 3D image generation method according to the number of pixels C of the 3D image to be generated, it is possible to obtain a 3D image with a small difference in resolution between left and right images.

Third Embodiment
FIGS. 9A, 9B, and 9C are schematic structural views of the 3D imaging apparatus of the present embodiment.

In the following, the third embodiment will be described on the assumption of a portable terminal device consisting of a single housing, but in addition to electronic devices having small cameras such as folding type portable terminals and digital still cameras (DSCs). The same applies to the case of wearing.

As shown in FIGS. 9A and 9B, the 3D camera 204 including the first camera 202 and the second camera 203 is disposed on the back of the display element 205 of the rectangular casing 201 of the mobile terminal device.

FIG.9 (c) shows the block block diagram of a portable terminal device. The first camera 202 and the second camera 203 are controlled by the camera control unit 208, and store the photographed image in the storage unit 209 or display it using the display element 205.

The distance information measuring means 210 measures distance information of the subject from the images of the first camera 202 and the second camera 203.

FIGS. 10A and 10B show a general 3D image capturing method in the case where there is a long distance object 211 and a short distance object 212. FIG. 10A shows the case where both the far-distance object 211 and the short-distance object 212 are behind the virtual projection plane (or 3D display screen). On the other hand, FIG. 10B shows the case where the short distance object 212 is near the 3D camera 204, the long distance object 211 is behind the virtual projection plane, and the short distance object 212 is in front. The parallax angle range D2 of the subject in FIG. 10 (b) is wider than the parallax angle range D1 of the subject in FIG. 10 (a).

As shown in FIG. 10B, when the short-distance object 212 is very close, the parallax is very large in C2L-C2R, so in this state, it is too close to the 3D display screen to make stereoscopic viewing difficult. .

For example, the parallax (C2L-C2R) of the short distance object 212 can be reduced by shifting the horizontal position of the left and right images as in Patent Document 4, but at this time the parallax (C1L-C1R) of the long distance object 211 It may increase in the negative direction, making it impossible to view stereoscopically.

This is because the difference in parallax (ΔCL−ΔCR) between the near distance object 212 and the long distance object 211 is large.

Therefore, the following 3D image generation method is adopted.

If the distance of the closest subject 212 closest to the camera is equal to or greater than a certain value, a 3D image is obtained with the left and right images of the first camera image and the second camera image. This will be referred to as the long distance imaging mode.

As shown in FIG. 10A, the parallax difference between two objects | ΔBL−ΔBR | is small, and the parallax angle range D1 is narrow, so that both the long distance object 211 and the near distance object 212 can be comfortably viewed stereoscopically. it can.

For example, when the base length of the camera is 6.5 cm, if the distance of the closest subject is 1 m or more, the parallax angle can be within ± 1 degree, and a comfortable 3D image can be obtained. .

Next, when the distance of the near distance object 212 closest to the camera is less than a certain value (for example, 1 m), it is called a short distance shooting mode, and the following 3D image generation method is adopted.

This is a method of calculating or estimating distance information (depth information) for each pixel or minute area from the first camera image and the second camera image, and generating a 3D image from the distance information, as described in, for example, Patent Document 7 ing.

FIG. 11 shows a 3D image generation method in the short distance shooting mode and the long distance shooting mode. Further, FIG. 13 shows a flowchart of 3D imaging.

When the closest short-distance subject distance L is larger than L0, the long-distance shooting mode is set, and a 3D image is obtained from the left and right images [1] and [2].

On the other hand, when the short-distance subject distance L is smaller than L0, the short-distance shooting mode is selected, and from left and right images [3] and [4], correlation calculation of image processing is performed on each pixel (or minute block). By providing the correspondence, the parallax information (distance information) [5] of each pixel is calculated. The measurement of the parallax information (distance information) is performed by the distance information measuring means 210. Further, left and right images [6] and [7] are obtained from the first camera image [3] (or the second camera image [4]) and the parallax information [5].

In the short distance shooting mode, the sense of depth can be made smaller than the actual one by multiplying the calculated distance information by a certain constant value (a) smaller than 1 in order to make the parallax angle range narrower than the actual.

For example, as shown in FIG. 12B, assuming that the parallax angle obtained from the distance information [5] of each pixel (x, y) calculated from [3] and [4] is A (x, y), If A ′ (x, y) = A (x, y) × a (0 <a <1), the parallax angle range can be made narrower than in reality.

That is, the parallax angle difference | ΔCL′−ΔCR ′ | in the short distance shooting mode of FIG. 12 (a) corresponds to the short distance object of FIG. 10 (b) in the images [1] and [2] of the long distance shooting mode. It can be made smaller than the difference | ΔCL−ΔCR | of the parallax angle of the long distance object.

As a result, the parallax angle range D3 becomes narrower than the parallax angle range D2, and even when the short distance object is very close, the fluctuation of the parallax angle is not too large, and comfortable stereoscopic vision can be achieved.

Next, a method of measuring the distance of the closest near object will be described.

The user may switch between the long distance shooting mode and the short distance shooting mode, but in order to switch automatically, the 3D image pickup apparatus has means for measuring the distance of the closest subject. is necessary.

As a method of measuring the distance of a short distance object, a general distance measuring device using a reflection time of radar or infrared rays may be used, but since the portable terminal device becomes expensive and the size increases, here A method of estimating from the images of the camera 202 and the second camera 203 is adopted. This eliminates the need for additional components for the distance measuring means.

For example, in the distance information measurement means 210, if the corresponding pixels (or blocks) of the left and right camera images are known, the distance of the pixels (blocks) can be estimated. It can be estimated that the closest subject among the pixels is the short distance subject.

In addition, since it takes processing time to calculate the corresponding pixels of all the left and right images, distance information may be calculated for pixels (or blocks) of several points in the photographed image.

In addition, since the subject is generally considered to be concentrated at the center of the image, the distance of the closest subject may be estimated from the distance information of the pixels (or blocks) near the center.

As described above, by combining the short distance 3D shooting mode in which the parallax angle is adjusted by the distance information measurement means in addition to the conventional long distance 3D shooting mode, both the 3D effect and the visibility can be achieved with the inexpensive and compact camera configuration. 3D photography is possible.

Embodiment 4
FIGS. 14 (a), (b) and (c) are schematic structural diagrams of the 3D imaging apparatus of the present embodiment.

In the following, the fourth embodiment will be described on the assumption of a portable terminal device consisting of a single housing, but in addition, electronic devices having small cameras such as folding type portable terminals and digital still cameras (DSCs) are described. The same applies to the case of wearing.

As shown in FIGS. 14A and 14B, a binocular 3D camera 304 including a first camera 302 and a second camera 303 is disposed on the back of the display element 305 of the rectangular casing 301 of the portable terminal device. ing.

The first camera 302 and the second camera 303 are composed of a camera sensor 307 and a lens 306.

FIG. 14C shows a block diagram of the portable terminal device. The first camera 302 and the second camera 303 are controlled by the camera control unit 308, store the captured image in the storage unit 309, and display the captured image using the display element 305.

The display element 305 is, for example, a naked-eye 3D liquid crystal having a barrier liquid crystal, and can switch between horizontal and vertical display of a 3D image by switching the barrier direction to vertical and horizontal.

In addition, even in the case of a 3D display device using electronic shutter glasses, it is possible to display horizontally and vertically in a 3D image.

Note that if there is no need to display a 3D image with a display element, vertically long 3D display may not be possible. Alternatively, 3D display itself may not be possible.

The distance information extraction unit 310 has a function of extracting distance information of an object from images of the first camera 302 and the second camera 303.

Furthermore, this portable terminal incorporates an attitude sensor 311 including an acceleration sensor, an azimuth sensor, and the like, and can detect the attitude of the terminal.

The imaging in which the long side of the display element is horizontal is referred to as a horizontal 3D imaging mode, and the imaging in which the long side of the display element is vertical is referred to as a vertical 3D imaging mode.

The posture sensor 311 detects whether the long side of the display element is horizontal or vertical, and automatically determines whether to set the horizontal 3D shooting mode or the vertical 3D shooting mode.

In the case where the posture sensor 311 is not provided, the user may select the horizontal 3D photographing mode or the vertical 3D photographing mode, so the posture sensor 311 may not be provided.

FIG. 15A shows an outline of the horizontal 3D shooting mode, and FIG. 15B shows an outline of the vertical 3D shooting mode.

The long sides of the camera sensors 307 of the first camera 302 and the second camera 303 are parallel to the long sides of the display element.

In the horizontal shooting mode, a 3D image is obtained by converting the first camera image and the second camera image into left and right images as in the prior art.

As shown in FIG. 15A, in reproduction display or preview display of a 3D image, it becomes a 3D image having horizontal parallax in the horizontal direction, that is, in the long side direction of the display element.

On the other hand, in the vertical shooting mode, the left and right images are 3D images having left and right parallax in the short side direction of the display element.

The vertical 3D image having left and right parallax in the short side direction can not be obtained only by using the first camera image and the second camera image as the left and right images. Next, a method of generating a vertical 3D image will be described.

FIG. 16 shows a generation procedure of horizontal 3D images and vertical 3D images.

In the horizontal 3D imaging mode, a 3D image is obtained from the left and right images [1] and [2].

On the other hand, in the vertical 3D shooting mode, each pixel (or minute block) is made to correspond to each other by using correlation calculation of image processing or the like from the vertically oriented left and right images [3] and [4]. Parallax information (= distance information) [5] of (or small block) is calculated. Furthermore, from the first camera image [3] (or the second camera image [4]) and the parallax information [5], left and right images [6] and [6] having left and right parallax (BL-BR) in the short side direction of the camera image 7] get.

Although a method of generating a 3D image after calculating distance information from a twin-lens camera image is described in, for example, Patent Document 6, a short-side of the display element is obtained from a twin-lens camera disposed in the long side direction of the display element. The point of generating a 3D image having a parallax of direction (BL-BR) is the difference between the present invention and the conventional example.

Also in the horizontal 3D shooting mode, as in the vertical 3D shooting mode, a 3D image generation method using distance information may be used.

However, since the 3D image generation method using distance information takes a long time to process, it is preferable to generate a 3D image from a conventional twin-lens camera image in the horizontal 3D imaging mode.

FIG. 17 shows an example of a 3D imaging flowchart. In the case where the detection result of the posture sensor 311 is vertical, the mode is switched to the vertical 3D imaging mode, and in the case of horizontal installation, the mode is switched to horizontal 3D imaging mode.

In the case of landscape orientation, the 3D preview display image uses the first camera image and the second camera image.

On the other hand, in the case of portrait orientation, the 3D images generated in steps S302 and S303 are used as the 3D preview display image. At this time, it is necessary to switch the barrier liquid crystal of the parallax barrier liquid crystal to the mode of vertical display.

As described above, by combining the 3D image generation method with a normal twin-lens camera and the 3D image generation method from distance information and one-sided camera image, it is compact and inexpensive without physically changing the arrangement of the cameras. In addition, vertical 3D shooting and horizontal 3D shooting can be compatible.

Fifth Embodiment
Next, a preview display method in 3D imaging according to the fifth embodiment will be described.

The basic part of the 3D image generation method of the fifth embodiment is the same as that of the fourth embodiment, so the description will be omitted.

FIG. 18 shows an example of a 3D imaging flowchart of the fifth embodiment.

In the case where the detection result of the posture sensor 311 is vertical, the mode is switched to the vertical 3D imaging mode, and in the case of horizontal installation, the mode is switched to the horizontal 3D imaging mode.

The fourth embodiment differs from FIG. 17 in the method of preview display in the case of the vertical 3D shooting mode in step S312.

In the fourth embodiment, the preview display at the time of vertical 3D shooting uses the preview image generated from the 3D image generated from the first or second camera image and the distance information, but it takes time to generate the 3D image using distance information Therefore, there is a problem that the start time of the preview display is delayed or the display frame rate is slow.

Therefore, here, the preview display is performed using a 2D image of the first or second camera image. The 3D barrier liquid crystal is not in 3D display mode but in 2D display mode.

Thereby, the problem of the delay in the preview display at the time of the vertical 3D shooting can be solved.

In addition, it is desirable to add the character display or mark display etc. which show that it is in 3D display preview in a 2D display screen so that a user can distinguish mere vertical 2D photography and vertical 3D photography mode clearly. .

FIG. 19 shows an example of a 3D imaging flowchart using another preview display.

The difference between FIG. 19 and FIG. 18 is the method of preview display in the vertical 3D shooting mode in step S321.

In step S321, a pseudo 3D image is previewed from the first camera image or the second camera image. Here, the pseudo 3D image refers to the following image.

As shown in FIG. 20, the left and right images ([8] and [9]) are two images that differ in the amount of horizontal shift (C) of the 2D image ([3]) of the first camera image or the second camera image. I assume. This is called a pseudo 3D image.

As a result, an image with parallax can be generated, and a preview image different from the preview at 2D shooting can be displayed. This makes it easy for the user to recognize whether 3D shooting or 2D shooting and distinguish between 2D shooting and 3D shooting. There is no need to display the

As described above, since the preview display method at the time of vertical shooting does not require generation of a 3D image, preview display can be performed in a short time. It is not necessary to display characters or marks in the 2D display screen to distinguish it from the 2D display preview.

Sixth Embodiment
Next, a parallax adjustment method in 3D imaging of the sixth embodiment will be described.

The basic part of the 3D image generation method of the sixth embodiment is the same as that of the fourth and fifth embodiments, so the description will be omitted.

21 (a) and 21 (b) are diagrams showing a 3D image generation method according to the sixth embodiment.

21A shows a 3D image at the time of horizontal shooting, and FIG. 21B shows a 3D image at the time of vertical shooting.

The parallax of the left and right images can be set independently at the time of horizontal shooting and vertical shooting. It is assumed that the subject shoots a short distance subject such as a person more frequently in the vertical shooting than in the horizontal shooting.

At the time of vertical shooting, the relative horizontal distance between the left and right images is adjusted in advance to a predetermined value so that the parallax angle (or parallax C2) of the subject at a short distance does not become too large. That is, the parallax setting value is made to be for the short distance. As a result, a comfortable parallax 3D image can be obtained even in close-up shooting in vertical shooting.

On the other hand, in the case of horizontal shooting, assuming that there are many far-distance objects, the parallax setting value is used for long-distance so that the parallax angle (or parallax C1) does not become too small.

As described above, a 3D image of optimal parallax for vertical shooting and horizontal shooting can be obtained.

Seventh Embodiment
22 (a), (b) and (c) are schematic structural views of the 3D imaging apparatus of the present embodiment.

In the following, the seventh embodiment will be described on the assumption of a portable terminal device having a single housing, but in addition, it is an electronic device having a small camera such as a folding portable terminal or a digital still camera (DSC). The same applies to the case of wearing.

As shown in FIGS. 22A and 22B, a 3D camera 404 including a first camera 402 and a second camera 403 is disposed on the back surface of the display element 405 of the rectangular casing 401 of the portable terminal device.

FIG.22 (c) shows the block block diagram of a portable terminal device. The first camera 402 and the second camera 403 are controlled by the camera control unit 408, and save the photographed image in the storage unit 409 or display it using the display element 405.

The first camera 402 and the second camera 403 are composed of a sensor 407 and a lens 406. However, the angle of view of the second camera 403 is larger than the angle of view of the first camera 402. The camera control means 408 synchronizes the imaging range of the first camera image and the second camera image with the imaging time by adjusting the scan range and the scan speed for the image acquired by the sensor 407 of the second camera 403. Further, the camera control unit 408 obtains a 3D image with the first and second camera images as left and right images.

Hereinafter, adjustment of the scan range by the camera control unit 408 will be described with reference to FIG. The camera control unit 408 limits the scan range in the vertical direction between the position A1 and the position A2 in FIG. 23 so that the second camera 403 scans only the range of the same angle of view as the angle of view of the first camera 402. That is, the camera control unit 408 does not scan the range not captured by the first camera 402 in the image acquired by the sensor 407 of the second camera 403.

Next, adjustment of scan speed by the camera control unit 408 will be described with reference to FIG. The camera control means 408 controls the second camera 403 so that the time for scanning the adjusted range with respect to the second camera 403 is the same time T2 as the time T1 for scanning all of the images of the sensor 407 of the first camera 402. Among the images acquired by the sensor 407, the speed of scanning the limited range is adjusted. The adjustment of the scan speed is performed by changing the sampling period of the pixels of the second camera 403, the blanking period, or the frequency of the clock signal input to the second camera 403.

As described above, the camera control unit 408 matches the vertical angle of view from the start to the end of the scan with two images acquired at the same time by the first camera 402 and the second camera 403 having different angles of view. Scan the same angle of view image at the same speed. Therefore, the first camera image of the first camera 402 and the second camera image of the second camera 403 synchronized with the angle of view of the first camera image and the imaging time can be obtained.

In the above description with reference to FIG. 23, the imaging time of the first camera 402 and the imaging time of the second camera 403 are matched by reducing the scanning speed of the second camera 403 having a large angle of view. However, the present invention is not limited to this method, and the imaging time for the same angle of view of two camera images may be synchronized by increasing the scanning speed of the first camera 402 with a small angle of view.

Further, in the present embodiment, even if only one camera has the optical zoom function, the imaging time can be similarly adjusted. For example, if the first camera 402 performs an optical zoom, the angle of view of the first camera 402 decreases. At this time, the camera control unit 408 adjusts the scan range and the scan speed of the second camera 403 in accordance with the change of the optical zoom magnification of the first camera 402 to obtain the imaging ranges and imaging times of the two camera images. It can synchronize.

For example, after adjusting the scan range and scan speed in the second camera 403 in accordance with the angle of view of the first camera 402 in a state where the optical zoom magnification of the first camera 402 is 1 ×, the optical zoom magnification of the first camera 402 is When it is n, the camera control unit 408 changes the frequency of the clock signal input to the second camera 403 to 1 / n. In this case, the camera control means 408 can synchronize the imaging range of two camera images with the imaging time only by adjusting only the scan range of the second camera 403 to 1 / n in accordance with the magnification.

Note that the first camera 402 and the second camera 403 may have a mechanism to synchronize externally, and may be configured to synchronize the imaging start time.

As described above, according to the present embodiment, when the first camera image and the second camera image for obtaining a 3D image are captured, the angle of view of the first camera and the angle of view of the second camera are different. For example, the scan range and scan speed for the image acquired by the sensor of the camera with the larger angle of view are adjusted so that the imaging range and imaging time of the first camera image and the second camera image are synchronized. Therefore, it is possible to electrically obtain left and right images for a 3D image without performing an operation for changing the angle of view with respect to the camera. For this reason, it is possible to obtain 3D images of a moving subject.

Further, even if the angle of view of one of the cameras is changed, the synchronization of the imaging time of the first camera image and the second camera image may be performed by changing the frequency of the clock signal. Therefore, it is possible to cope with the change of the angle of view without changing the setting such as the blank period of the camera.

Although the invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.

The present application relates to Japanese Patent Application filed on March 17, 2011 (Japanese Patent Application No. 2011-058828), Japanese Patent Application filed on March 17, 2011 (Japanese Patent Application No. 2011-058827), and Japanese Patent Application filed on March 17, 2011 This application is based on Japanese Patent Application (Japanese Patent Application No. 2011-058832), the contents of which are incorporated herein by reference.

The present invention is useful as a 3D camera, and can be used in various electronic devices having cameras such as mobile phones, mobile terminals, digital still cameras, and so on.

101 Case 102 First Camera 103 Second Camera 104 3D Camera 105 Display Element 106 Lens 107 Sensor 108 Camera Control Means 109 Zoom Control Signal 110 Storage Means 111 Third Camera 112 Fourth Camera 201 Housing 202 First Camera 203 Second Camera 204 3D camera 205 Display element (Display section)
206 lens 207 sensor 208 camera control means 209 storage means 210 distance information measuring means 211 long distance object 212 near distance object 301 housing 302 first camera 303 second camera 304 3D camera 305 display element 306 lens 307 camera sensor 308 camera control means 309 storage means 310 distance information extraction unit 311 attitude sensor

Claims

With two cameras with different optical axes,
A distance measuring unit that measures a distance to an object from parallax of two camera images captured by each of the two cameras;
A first 3D image generation unit that generates a 3D image from the two camera images;
A second 3D image generation unit configured to generate a 3D image based on the distance to the subject measured by the distance measurement unit and a camera image captured by one of the two cameras;
A 3D imaging apparatus that switches between generation of a 3D image by the first 3D image generation unit and generation of a 3D image by the second 3D image generation unit according to a shooting state based on the positions of the two cameras with respect to the subject .
The 3D imaging device according to claim 1, wherein
The two cameras are a first camera with an optical zoom function and a second camera of a single focus type,
When the zoom magnification by the first camera is smaller than a predetermined magnification, the first 3D image generation unit generates a 3D image from the optical zoom image of the first camera and the electronic zoom image of the second camera,
When the zoom magnification is larger than the predetermined magnification, the second 3D image generation unit generates a 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by the first camera. 3D imaging device to generate.
The 3D imaging device according to claim 1, wherein
The two cameras are a third camera and a fourth camera having a smaller number of pixels than the third camera.
When the number of pixels of the 3D image to be generated is smaller than a predetermined number of pixels, the first 3D image generation unit generates a 3D image from the camera image captured by the third camera and the camera image captured by the fourth camera And
When the number of pixels of the 3D image to be generated is larger than the predetermined number of pixels, the second 3D image generation unit measures the distance to the subject measured by the distance measurement unit and the camera image captured by the third camera And a 3D imaging device that generates 3D images.
The 3D imaging device according to claim 1, wherein
The two cameras are a third camera and a fourth camera having a smaller number of pixels than the third camera.
At the time of 3D moving image shooting, the first 3D image generation unit generates a 3D image from the camera image captured by the third camera and the camera image captured by the fourth camera,
At the time of 3D still image shooting, the second 3D image generating unit generates a 3D image based on the distance to the subject measured by the distance measuring unit and the camera image captured by the third camera apparatus.
The 3D imaging device according to claim 1, wherein
In the long-distance shooting mode in which the distance to the subject measured by the distance measuring unit is equal to or more than a predetermined value, the first 3D image generating unit is a camera image captured by one of the two cameras and the other of the two cameras Generates a first 3D image from the camera image captured by
In the short distance shooting mode in which the distance to the subject measured by the distance measurement unit is less than the predetermined value, the second 3D image generation unit determines the distance to the subject measured by the distance measurement unit, and the two cameras Generating a second 3D image based on the camera image captured by one of the
The 3D imaging device in which the parallax angle difference between the first object and the second object at different parallax angles in the second 3D image is smaller than the parallax angle difference between the first object and the second object in the first 3D image.
The 3D imaging apparatus according to claim 5, further comprising: a display unit configured to output the first 3D image and / or the second 3D image.
The 3D imaging apparatus according to claim 5 or 6, wherein
The distance measurement unit calculates the closest distance of the third object closest to the 3D imaging device,
When the closest distance is equal to or more than the predetermined value, the far-field shooting mode is set.
The 3D imaging apparatus in which the short distance imaging mode is set when the closest distance is less than the predetermined value.
The 3D imaging device according to claim 1, wherein
And
A rectangular display element disposed in the housing;
The two cameras are arranged side by side in the long side direction of the display element,
In the vertical 3D shooting mode in which the long side of the display element is vertical, the distance measuring unit measures the distance from the parallax of the short side of the display element of the two camera images captured by each of the two cameras to the object And the second 3D image generation unit generates a 3D image based on the distance to the subject measured by the distance measurement unit and the camera image captured by one of the two cameras. .
The 3D imaging device according to claim 8, wherein
In the horizontal 3D imaging mode in which the long side of the display element is horizontal, the first 3D image generation unit generates a 3D image from camera images captured by each of the two cameras.
The 3D imaging apparatus according to claim 8 or 9, wherein
It has an attitude sensor that detects whether the long side of the display element is horizontal or vertical,
A 3D imaging device that switches to the vertical 3D imaging mode or the horizontal 3D imaging mode according to the detection result of the attitude sensor.
The 3D imaging apparatus according to any one of claims 8 to 10, wherein
The display device has a 3D display function and displays a 3D preview in the horizontal 3D shooting mode, and in the vertical 3D shooting mode, the display device is a 2D image using a camera image of one of the two cameras. 3D imaging device that performs preview display by.
The 3D imaging device according to claim 11, wherein
The display device is a 3D imaging device that performs the preview display with a pseudo 3D image in which parallax is added in a short side direction of the display device from the camera image of one of the two cameras in the vertical 3D imaging mode.
The 3D imaging apparatus according to any one of claims 8 to 12, wherein
A 3D imaging apparatus capable of independently setting parallax settings of left and right images of the horizontal 3D imaging mode and the vertical 3D imaging mode.
The 3D imaging device according to claim 13, wherein
The 3D imaging device, wherein the parallax setting value of the vertical 3D imaging mode is for a short distance than the horizontal 3D imaging mode.
With the first camera,
A second camera having an optical axis different from the optical axis of the first camera and having a larger angle of view than the first camera;
A camera control unit that controls the second camera to pick up a camera image of a shooting range equal to the angle of view of the first camera in synchronization with the shooting time of the first camera;
A 3D image generator configured to generate a 3D image from the camera image of the first camera and the camera image of the second camera;
3D imaging device with.
The 3D imaging device according to claim 15, wherein
The camera control unit changes a frequency of a clock signal input to the second camera to synchronize an imaging time of a camera image of the second camera with an imaging time of the first camera.