JP2006345246A - Portable phone with three-dimensional photography function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a slide type portable phone with a three-dimensional (3D) photography function capable of increasing a parallax between cameras and suited to generation of 3D images. <P>SOLUTION: The portable phone with the 3D photography function is constituted of a first casing 1 on which a first camera 3L and a display part 6 are arranged and a second casing 2 on which the first casing is slidably mounted and a second camera 3R is arranged. Where the first camera 3L is arranged on the sliding-direction end part of the first casing 1, the second camera 3R is arranged on the end part of the second casing, in the direction opposite to the sliding direction of the first camera 3L, and when the first casing 1 is slid, a distance between the first camera 3L and the second camera 3R is increased. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a mobile phone having a camera shooting function, and more particularly to a slide type mobile phone capable of shooting and displaying a stereoscopic image.

FIG. 3 is an external view showing a configuration of a mobile phone having a conventional camera photographing function described in Non-Patent Document 1 (NTT-DoCoMo “Instruction Manual FOMA D901i”).
As shown in the figure, in this mobile phone, a first casing 100 in which an LCD (Liquid Crystal Display) 102 for image display is arranged is slidably attached to a second casing 200 in which a battery is stored. ing.
3A is a view when viewed from the front side (the first housing side where the operator of the mobile phone is present), and FIG. 3B is the back side (the second housing side). It is a figure when it sees from.

As can be seen from FIG. 3A, a self-portrait camera 101 for videophone is arranged in the first casing 100 in which the LCD 102 is arranged.
The direction of the self-portrait camera 101 (that is, the shooting direction) is fixed, and the self-portrait camera 101 can shoot only the LCD display side (that is, the side where the operator of the mobile phone is present).
The LCD 101 is for a normal 2D (2-Dimension) display.
Further, as can be seen from FIG. 3B, a high pixel camera 201 for moving image / still image shooting is arranged on the back side (back side) of the second casing 200.
The direction (photographing direction) of the high pixel camera 201 is also fixed, and only the back side of the mobile phone (that is, the front side of the operator of the mobile phone) can be taken.

Note that when shooting with the high pixel camera 201 (shooting in a so-called digital camera mode), it is possible to perform shooting using the LCD 101 arranged in the first housing 100 as a viewfinder.
As described above, the mobile phone D901i described in Non-Patent Document 1 includes a camera (self-portrait camera 101 or high-pixel camera 201) for taking a 2D image in the same manner as a general camera. Only shooting and display can be performed.

Further, as described in Patent Documents 1 to 4, there is an idea of recording a stereoscopic image by simultaneously capturing two images using parallax.
For example, in Patent Document 1 (Japanese Patent Application Laid-Open No. 2005-6266), two lenses are provided in an integrated video camera or mobile phone so that they can be recorded independently and simultaneously. It is described that a stereoscopic image can be viewed using a 3D reproduction glass, which is dedicated glasses, so that the viewer can view the images simultaneously.
In Patent Document 2 (Japanese Patent Laid-Open No. 2004-242092), two digital cameras for performing stereoscopic shooting are arranged in the longitudinal direction of an integrally configured terminal (that is, a mobile phone terminal), thereby obtaining parallax, A camera-equipped mobile phone configured to display a three-dimensional image with a stereoscopic effect is described.

In Patent Document 3 (Japanese Patent Application Laid-Open No. 2004-120611), two cameras for acquiring a stereoscopic image are incorporated in an integrally configured housing, and one of the two cameras is rotated. A portable device with a function of simultaneously photographing multiple images is described.
In Patent Document 4 (Japanese Patent Application Laid-Open No. 2003-51872), a first casing on which a first camera is mounted and a second casing on which a second camera is mounted can be folded by a rotating shaft. A portable communication device configured to generate a stereoscopic image by opening a second housing relative to a first housing to gain parallax between the first camera and the second camera is described. ing.
Patent Document 4 discloses an idea for rotating a camera to switch between three-dimensional self-portrait and three-dimensional self-portrait, and 3D (3-dimension: three-dimensional) display (that is, three-dimensional image display) on the same terminal. The idea of mounting the device at the same time is also described.
Japanese Patent Laying-Open No. 2005-6266 (FIG. 3, paragraph 0004) Japanese Patent Laying-Open No. 2004-242092 (FIG. 2, paragraph 0030) JP 2004-120611 A (FIG. 1, paragraph 0022) JP 2003-51872 A (FIGS. 5, 6, paragraph 0039, paragraph 0013, paragraph 0008) NTT-DoCoMo “Instruction Manual FOMA D901i”

A conventional general mobile phone has only one camera, and the conventional mobile phone shown in Non-Patent Document 1 has two cameras, but these two cameras are fixed differently. Since it was fixedly incorporated so as to photograph the direction (front side and back side), it was not possible to photograph a stereoscopic image.
The mobile phones described in Patent Documents 1 to 3 describe that two cameras are used to generate or view a stereoscopic image, but both have two cameras in an integrally configured housing (mobile phone). It is what is arranged.

  Patent Document 4 states that “a first casing on which a first camera is mounted and a second casing on which a second camera is mounted can be folded. Although it is shown that by opening the second housing, the parallax between the first camera and the second camera is obtained, and a stereoscopic image is obtained. There is no description that suggests that "the parallax between the first camera and the second camera is earned by sliding".

In general, when two-dimensional photography is performed using two cameras, the two cameras operate independently from each other, causing the following problems.
* Since the right-eye image and the left-eye image are captured at different timings, when a moving object or a changing object is photographed stereoscopically, the captured image has a sense of incongruity.
* Since the right-eye camera and the left-eye camera perform independent automatic image adjustment, the right-eye image and the left-eye image have different brightness, color tone, and focus position, resulting in a sense of discomfort.

In the mobile phones described in Patent Documents 1 to 4, although the stereoscopic method and the manual method for stereoscopic viewing of the captured stereoscopic image are described, the shadow timing of the right-eye camera and the left-eye camera is different, No image signal processing method is described for preventing a sensed image taken by the right-eye camera and the left-eye camera to perform an automatic image adjustment independently.
Note that Patent Documents 1 to 4 do not describe a method for storing a stereoscopic image, but a moving image compression / decompression method for reducing the amount of data of a stereoscopic image and storing it is a code dedicated to a stereoscopic image. Processing is required.

FIG. 4 is a block diagram showing a configuration of an image processing system in a conventional mobile phone with a stereoscopic shooting / playback function, and is compared with a block diagram showing a configuration of an image signal processing system in a mobile phone with a stereoscopic shooting function according to the present invention described later. It is shown.
In the figure, 300R is a right-eye camera, and the right-eye camera 300R includes a right-eye AF (Auto Focus) lens driving unit 301R, a right-eye image sensor 302R, a right-eye image signal processing unit 303R, and a right-eye camera. The image sensor control unit 304R for the right eye, the imaging condition processing unit 305R for the right eye, and the AF control unit 306R for the right eye.
Reference numeral 300L denotes a left-eye camera. The left-eye camera 300L includes a left-eye AF lens driving unit 301L, a left-eye image sensor 302L, a left-eye image signal processing unit 303L, and a left-eye image sensor control unit. 304L, a left-eye imaging condition processing unit 305L, and a left-eye AF control unit 306L.

Also, 400R is an image compression unit for the right eye, 450R is an image expansion unit for the right eye, 600R is an image display unit for the right eye, 400L is an image compression unit for the left eye, 450L is an image expansion unit for the left eye, and 600L is the left An eye image display unit 500 is a storage memory unit.
In the figure, double-line arrows indicate the flow of image signals, thin-line arrows indicate the flow of control information, and thick-line arrows indicate the flow of compressed image information.
A range surrounded by a wavy line (that is, the right-eye camera 300R or the right-eye camera 300L) indicates components included in a general camera.

In a conventional mobile phone with a stereoscopic shooting / playback function, two cameras for performing stereoscopic shooting (that is, a right-eye camera and a left-eye camera) operate independently from each other. Occurs.
* The image sensor control unit is considered to be independent for the right eye and the left eye. As a result, the exposure timing for the right eye image differs from the exposure timing for the left eye image, especially when moving objects are photographed. In this case, the position where the subject is reflected changes due to the deviation of the exposure timing, which causes a sense of incongruity during stereoscopic viewing.
* The imaging signal processing unit that determines the exposure conditions of the camera is considered to be independent for the right eye and the left eye. As a result, the analog gains of the right eye image sensor and the left eye image sensor differ, so the right The brightness of the image for the left eye and the image for the left eye are different, and the exposure time of the image sensor for the right eye and the image sensor for the left eye is different, so the moving subject reflected in the image for the right eye and the image for the left eye is blurred. This may cause a sense of incongruity during stereoscopic viewing, such as being different.

* It is considered that the AF control unit that determines the focus position of the camera is independent for the right eye and the left eye. As a result, the focus position of the right eye camera and the focus position of the left eye camera are different, The blurring of the subject of the image for the eye and the image for the left eye is different, which causes a sense of incongruity during stereoscopic viewing.
* The image compression process is considered to be independent for the right eye and the left eye, and as a result, the compression conditions for the right eye image and the left eye image are different, resulting in differences in compression noise. This may cause discomfort during stereoscopic viewing.
* It is considered that the image expansion process is independent for the right eye and the left eye. As a result, the expansion conditions for the right eye image and the left eye image are different, resulting in differences in expansion timing, etc. It sometimes causes discomfort.

The present invention has been made to solve such a problem, and the first housing in which the first camera is arranged and the second housing in which the second camera is arranged can be slid freely. It is possible to increase the parallax between the first camera and the second camera by disposing the first and second cameras, and to provide a slide-type mobile phone with a stereoscopic shooting function suitable for generating a stereoscopic image. To do.
Another object of the present invention is to provide a mobile phone with a stereoscopic photographing function that can provide a high-quality stereoscopic image without feeling uncomfortable when the generated stereoscopic image is viewed.
In addition, the present invention further provides a mobile phone with a stereoscopic photographing function capable of simultaneously recording a stereo image when generating a stereo image, compressing / decompressing the stereo image, and speeding up the autofocus convergence. Objective.

  A mobile phone with a stereoscopic shooting / playback function according to the present invention includes a first housing in which a first camera and a display unit are disposed, and the first housing is slidably mounted. A mobile phone with a three-dimensional imaging function configured with a second casing disposed, wherein the first camera is disposed at an end of the first casing in a sliding direction, and the second camera Is arranged at the end of the second casing opposite to the sliding direction of the first camera, and when the first casing is slid, the first camera and the second camera The distance between the cameras is increased.

  In the mobile phone with a stereoscopic photographing / playback function according to the present invention, a first microphone is disposed in the vicinity of the first camera, and a second microphone is disposed in the vicinity of the second camera. is there.

  In the mobile phone with a stereoscopic shooting / playback function according to the present invention, the first camera and the second camera operating in synchronization are controlled under the same shooting conditions.

  The mobile phone with a stereoscopic photographing / playback function according to the present invention connects two image signals output from the first camera and the second camera operating in synchronism to each frame to connect one image. It converts to a signal.

  In addition, the mobile phone with a stereoscopic shooting / playback function according to the present invention estimates the distance to the subject based on the difference between the subject positions in the photographed images of the first camera and the second camera operating in synchronization. The focus position is automatically adjusted.

  According to this invention, the first camera is disposed at the end of the first casing in the sliding direction, and the second camera is on the opposite side of the second casing from the sliding direction of the first camera. Since the distance between the first camera and the second camera is increased when the first casing is slid, the first casing is thrust. Thus, the parallax between the first camera and the second camera can be increased, and an image suitable for generating a stereoscopic image can be easily taken.

  Further, according to the present invention, since the first microphone is disposed in the vicinity of the first camera and the second microphone is disposed in the vicinity of the second camera, stereo recording can be performed simultaneously with stereoscopic shooting. Become.

  In addition, according to the present invention, the first and second cameras operating in synchronism are controlled under the same shooting conditions, so that higher-quality stereoscopic image shooting without a sense of incongruity becomes possible.

  In addition, according to the present invention, two image signals output from the first camera and the second camera are connected for each frame and converted into one image signal. It is possible to compress / decompress a stereoscopic image using an expansion / decompression process.

  Further, according to the present invention, the distance to the subject is estimated based on the difference between the subject positions in the captured images of the first camera and the second camera, and the focus position is automatically adjusted. In any case of stereoscopic shooting, convergence of autofocus can be accelerated.

Embodiment 1 FIG.
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
In addition, between each figure, the same code | symbol represents that it is the same or equivalent.
FIG. 1 is a perspective view showing an external configuration of a mobile phone with a stereoscopic photographing function according to the present invention.
In the figure, 1 is a first casing (display-side casing), 2 is a second casing (battery-side casing), and 3L is at the end of the first casing (display-side casing) 1. A left-eye camera (first camera) 3R that is arranged and rotatable about 180 degrees, and 3R is arranged at the end of the second casing (display-side casing) 2 and can rotate about 180 degrees The camera (second camera), 4L is the sound collecting hole of the left microphone (first microphone) arranged integrally with the left-eye camera (first camera) 3L, and 4R is the right-eye camera (second camera). The sound collecting hole of the right microphone crofun (second microphone) arranged integrally with the camera 3R, 5L is the sound emitting hole of the left speaker arranged in the first housing (display housing) 1, and 5R is the first sound collecting hole. A sound emitting hole of the right speaker arranged in one housing (display-side housing) 1, 6 is a table arranged in the first housing (display-side housing) 1 Part (stereoscopic display panel), 7 is an operation key arranged in the first casing (display-side casing 1), and 8 is an operation key arranged in the second casing (battery-side casing) 2. .

A first housing (display-side housing) 1 is slidably attached to a second housing (battery-side housing) 2.
FIG. 1A shows a state when the first casing (display-side casing) 1 is not slid and overlaps the second casing (battery-side casing) 2. FIG. 1B shows a state in which the first housing (display-side housing) 1 is slid leftward with respect to the second housing 2.
That is, when the main body of the mobile phone is constituted by the first casing (display-side casing) 1 and the second casing (battery-side casing) 2, FIG. FIG. 1 (b) shows a state when the main body is opened.

The cellular phone with a stereoscopic photographing function according to the present embodiment has a right speaker (inside the first speaker 1 corresponding to the sound emitting hole 5R of the right speaker) in a state where the main body shown in FIG. And the left microphone (arranged integrally with the left-eye camera 3L corresponding to the sound collecting hole 4L of the left microphone) and can be used for voice calls.
When it is necessary to input the destination dial number, the main body is opened (that is, the first casing 1 is slid to the left), and is placed in the second casing (battery side casing) 2. It is possible to input a destination dial number using the operation keys 8.
Similarly, when it is used as a videophone, when dial input is not necessary, the main body is closed (that is, the first casing is overlaid on the second casing 2), the right speaker, the left speaker. Communication can be performed using the left microphone, the left-eye camera 3L, and the display panel 6 (2D display mode).

When the user (photographer) is photographed by the camera, an image photographed by the left-eye camera 3L arranged at the end of the first housing 1 is displayed on the display panel 6 (2D display mode). Shooting can be performed while displaying.
When photographing a distant place with the camera, the right-eye camera 3R is rotated by rotating the right-eye camera 3R disposed at the end of the second housing 2 so that photographing in front of the photographer is possible. Photographing can be performed while displaying an image to be photographed on the display panel 6 (2D display mode).
In this way, what can be done with a conventional mobile phone can be realized as a whole.

In the mobile phone with a stereoscopic shooting function according to the present embodiment, when performing stereoscopic shooting of itself, the first housing (display-side housing) 1 is slid to open the main body, and the left-eye camera 3L By shooting with the right-eye camera 3R facing the front (that is, the direction in which the photographer is present), stereoscopic shooting using the parallax between the left-eye camera 3L and the right-eye camera 3R is possible.
The stereoscopic image being photographed can be photographed while being displayed in real time on the display panel 6 (3D display mode).

In this mobile phone, when performing stereoscopic shooting at a distance, the main body is slid, and the left-eye camera 3L and the right-eye camera 3R are directed toward the back (that is, in the direction in front of the photographer). Stereo shooting using the parallax between the camera 3L and the right-eye camera 3R is possible.
The stereoscopic image being photographed can be photographed while being displayed in the viewfinder (3D display mode) on the display unit 6 in real time.

FIG. 2 is a block diagram showing a configuration of an image signal processing system in the mobile phone with a stereoscopic shooting / playback function according to the present embodiment.
In the figure, 3L is a left-eye camera (first camera), and the left-eye camera 3L includes a left-eye AF lens driving unit 31L, a left-eye image sensor 32L, and a left-eye image signal processing unit 33L. It is configured. The left-eye camera 3L has a left-eye lens (not shown).
Reference numeral 3R denotes a right-eye camera (second camera). The right-eye camera 3R includes a right-eye AF lens driving unit 31R, a right-eye image sensor 32R, and a right-eye image signal processing unit 33R. Has been. The right-eye camera 3R includes a left-eye lens (not shown).
Also, 41 is an image sensor control unit, 42 is an image connection unit, 43 is an imaging condition processing unit, 44 is an AF (Auto Focus) control unit, 45 is an image compression unit, 46 is a storage memory unit, 47 is an image expansion unit, Reference numeral 48 denotes an image separation unit.

Reference numeral 6 denotes a display unit (stereoscopic display panel). The display unit 6 includes a right-eye image display unit 6R and a left-eye image display unit 6L integrated on a single display surface.
When the 2D display is performed, the display unit 6 displays the same image on the right-eye image display unit 6R and the left-eye image display unit 6L, so that the stereoscopic effect does not appear.
In 3D display (stereoscopic image display), different images using parallax between the right-eye camera 3R and the left-eye camera 3L are displayed on the right-eye image display unit 6R and the left-eye image display unit 6L. For this reason, a stereoscopic effect appears.
In the figure, microphones and speakers are not shown. In the figure, double-line arrows indicate the flow of image signals, thin-line arrows indicate the flow of control information, and thick-line arrows indicate compression. The flow of image information is shown.

The AF lens drive unit can change the distance between the lens that forms an optical image on the image sensor and the image sensor imaging surface in accordance with a control signal from the AF control unit. This is a mechanism for mechanically moving the lens in the optical axis direction by the actuator.
In the present embodiment, the right-eye AF lens driving unit 31R and the left-eye lens driving unit 31L are provided for the left-eye camera (first camera) 1 and the right-eye camera (second camera), respectively. It is provided separately to correspond.
This is because it is assumed that the positional relationship between the right-eye camera 3R and the left-eye camera 3L is spaced apart in order to earn parallax.
If the right-eye camera and the left-eye camera are arranged in the vicinity, it is possible to drive both the right-eye lens 3R and the left-eye lens 3L with one AF lens driving unit.

The image sensor converts an optical image formed on the imaging surface into an electrical signal according to control of the image sensor control unit, and outputs electrical image information. It is a part generally called a CCD sensor or a CMOS sensor.
However, in the case of a CMOS sensor, the sensor control unit and image signal processing are often integrated on a single silicon chip.
In the present embodiment, a right-eye image sensor 32R is disposed in the right-eye camera 3R, and a left-eye image sensor 32L is disposed in the left-eye camera 3L.

The image sensor control unit 41 generates a signal for controlling the operation timing of the right eye image sensor 32R and the left eye image sensor 32L.
For example, when a CCD image sensor is used as the image sensor, the following control is performed.
(1) Discharge of accumulated charge (unnecessary charge in the accumulation part is discharged before charge accumulation).
(2) Exposure and charge accumulation start (when light is input to the image plane pixel after the accumulated charge is discharged, a charge corresponding to the intensity is accumulated).
(3) Move the accumulated charge to the vertical transfer path (move the charge to the transfer path to output the charge accumulated in each pixel to the outside of the chip).
(4) The charge on the vertical transfer path is moved (the charge on the vertical transfer path is moved in the vertical direction by one pixel, and one column of the bottom pixel enters the horizontal transfer path).
(5) The charge on the horizontal transfer path is moved (the charge on the horizontal transfer path is moved in the horizontal direction by one pixel, and the charge at the end enters the AD converter).
(6) AD conversion execution / AD conversion data output (converts the amount of charge of one pixel into digital data). At this time, the full scale of AD conversion is determined by the analog gain value.
(7) Repeat from (5) for the number of horizontal pixels.
(8) Repeat from (4) for the number of vertical pixels.
In the control as described above, the time from (2) to (3) is a feeling during exposure.

The right-eye image signal processing unit 33R performs processing so that the output image of the right-eye image sensor 32R reproduces an appropriate color.
Similarly, the left-eye image signal processing unit 33L performs processing so that the output image of the left-eye image sensor 32L reproduces an appropriate color.
The output of the image sensor 32R for the right eye or the image sensor 32L for the left eye faithfully outputs the color temperature (light color) of the light source. However, if recorded as it is, an unnatural hue when viewing the reproduced image. It becomes.
Therefore, normally, the camera (that is, the right-eye camera 3R or the left-eye camera 3L) estimates the color temperature of the light source from the photographed image and processes it so as to have a shade under white light.
Although not related to the present embodiment, the image sensor 32R for the right eye and the image sensor 32L for the left eye correct the defective pixel of the image sensor, and the output of the image sensor such as a bitmap format or a YUV format. Convert to a data format that is easy to process.

The image connecting unit 42 connects the two images input from the right-eye image signal processing unit 33R and the left-eye image signal processing unit 33L into one image and outputs it.
For example, if the output of the image sensor 32R for the right eye and the image sensor 32L for the left eye is 640 dots wide × 480 dots high × 30 frames per second, the image connecting unit 42 is 640 dots wide × 480 dots high × 30 sheets per second. X2 systems of input images are converted into horizontal 1280 dots x vertical 480 dots x 30 images per second and output.
Note that, when stereoscopic imaging is not performed, the image connecting unit 42 outputs an input image from one of the right-eye image signal processing unit 33R or the left-eye image signal processing unit 33L as it is.

The imaging condition processing unit 43 examines the exposure state of the input image, determines whether or not the analog gain of the right eye image sensor 32R and the left eye image sensor 32L and the feeling at the time of exposure are appropriate, and if correction is necessary. A correction signal is output to the image sensor control unit 41.
The AF control unit 44 checks the contrast of the input image, determines whether or not the camera is correctly focused on the subject, and outputs a correction signal to the AF lens driving unit if correction is necessary.
At this time, high-speed and continuous focus control is performed by estimating the distance to the subject from the difference between the image position of the subject captured by the right-eye camera 3R and the image position of the subject captured by the left-eye camera 3L. Can do.

In a normal monocular camera, a plurality of images are taken while moving the lens position, and the lens position where the image with the highest contrast is obtained is determined as the optimum focus position.
Therefore, a plurality of exposures are required for one focusing, and the autofocus control is delayed.
Also, during moving image shooting, an image with a variable lens position cannot be acquired, so autofocus control cannot be performed.

The image compression unit 45 is, for example, JPEG compression processing or MPEG compression processing.
In the mobile phone with a stereoscopic shooting function according to the present embodiment, the image compression unit 45 does not need to perform processing in consideration of stereoscopic shooting, and may be a general one.
When recording stereo sound simultaneously with stereoscopic shooting, it is necessary to multiplex the audio compression unit and the compressed audio and compressed image. This can be applied to existing video cameras and mobile phones with a TV phone function. Since this is a function that has already been implemented, the description is omitted.
The storage memory unit 46 is a part for writing / reading data to / from a NandFlash built in a mobile phone or a miniSD card or MS-Duo card installed in a memory card slot of the mobile phone. Since this is a function implemented in an existing mobile phone, description thereof is omitted.

The image expansion unit 47 is, for example, JPEG expansion processing or MPEG expansion processing. The image expansion unit 47 of the mobile phone with a stereoscopic photographing function according to the present embodiment does not need to perform processing in consideration of stereoscopic photographing, and may be a general one.
When reproducing data in which stereo sound is recorded at the same time as a stereo image, it is necessary to separate compressed audio from the compressed image and to decompress the audio. Since this is a function already implemented in a mobile phone with a telephone function, the description thereof is omitted.

The image separating unit 48 separates one image sent from the image connecting unit 42 or the image decompressing unit 47 into two images and outputs them.
If the input image input to the image separation unit 48 is 1280 horizontal dots × 480 vertical dots × 30 frames per second, the image separation unit outputs horizontal 640 dots × vertical 480 dots × 30 frames per second × 2 lines of images.
If the input image is not a stereoscopic image, the image separation unit 48 outputs the input image as it is to both outputs.

The display unit 6 visually displays the input image information on the display surface of the display panel.
In the present embodiment, in order to stereoscopically display a stereoscopic image, a display panel capable of stereoscopically displaying two images captured using parallax is required, and the display unit 6 is a right-eye image. A display unit 6R and a left-eye image display unit 6L are provided.
The details of the stereoscopic display panel will be omitted because various methods have been announced as existing technologies.

The operation when performing normal shooting and when performing stereoscopic shooting using the mobile phone with the stereoscopic shooting function having the configuration shown in FIGS. 1 and 2 will be described.
When performing normal shooting, only one of the left-eye camera 3L and the right-eye camera 3R is operated.
For example, when normal imaging is performed using the left-eye camera 3L, the image connecting unit 42 does not use the output of the right-eye image signal processing unit 33R, and outputs the output image of the left-eye image signal processing unit 33L as it is. To do.

The photographing condition processing unit 43 analyzes the image of the left eye image signal processing unit 33L output from the image connecting unit 42, and outputs an exposure condition and a white balance condition for only the left eye image.
The image sensor control unit 41 adjusts the exposure time, the frame rate, the analog gain, and the like of the image sensor according to the exposure conditions analyzed only from the left-eye image, so that the left-eye image becomes the optimal exposure condition. The image sensor 32L is controlled.
The image signal processing unit 33L for the left eye performs color conversion processing appropriate for the image sensor output according to the white balance condition analyzed from only the image for the left eye, and maintains the optimal color reproducibility for the image for the left eye. Process the image signal.
The AF control unit 44 analyzes the image of the left-eye image signal processing output from the image connecting unit, and controls the left-eye AF lens driving unit so as to achieve the optimum contrast so as to focus the left-eye image. I do.

The left-eye AF lens drive unit 31L moves the lens (not shown) of the left-eye camera 3L according to the output of the AF control unit 44, and changes the focal position of the left-eye camera 3L.
The image compression unit 45 functions when saving a captured image, and the image of the left eye image signal processing output from the image connection unit is, for example, JPEG format if it is a still image, or MPEG format if it is a moving image. Output the compressed data.
The storage memory unit 46 stores the image data compressed by the image compression unit 45 in a NandFlash memory built in the mobile phone body, or a memory card such as a miniSD card or MS-Duo installed in a memory card slot of the mobile phone body. To do.
The image decompression unit 47 functions when reproducing a stored image, receives the compressed data stored in the storage memory unit, decompresses it, and restores the image before compression.
The image separation unit 48 selects either the image output from the image connection unit 42 or the image output from the image expansion unit, and if the selected image is taken normally, the image is used as the right-eye image. The data is directly output to the display unit 6R and the left-eye image display unit 6L.
The right-eye image display unit 6R and the left-eye image display unit RL display the image output from the image separation unit 48 on the display surface of the display unit (stereoscopic display panel) 6.

When performing stereoscopic shooting, both the left-eye camera 3L and the right-eye camera 3L are operated.
In this case, the image connecting unit 42 combines the output of the right-eye image signal processing unit 33R and the output image of the left-eye image signal processing unit 33L into a single image having twice the number of pixels.
For example, when the image for the right eye and the image for the left eye are images of horizontal 640 dots × vertical 480 dots, the output image of the image connecting unit is horizontal 1280 dots × vertical 480 dots.
At this time, the right-eye image sensor 32R and the left-eye image sensor 32L are driven by the single image sensor control unit 41 at the same timing, so the right-eye image and the left-eye image are the same at the same frame rate. The image is input to the image connecting unit 42 at the timing.

The photographing condition processing unit 43 analyzes the left / right composite image output from the image connecting unit 42 and outputs the exposure condition and white balance condition of the left / right composite image.
The image sensor control unit 41 adjusts the exposure time, the frame rate, the analog gain, and the like of the image sensor according to the exposure conditions analyzed from the left and right composite image, and the right eye image sensor so that the right and left composite image becomes the optimal exposure condition. 32R and left-eye image sensor 32L are controlled.
As a result, the exposure condition of the right eye image sensor 32R and the exposure condition of the left eye image sensor 32L are the same, and there is no mismatch between the exposure conditions of the right eye image and the left eye image.
The right-eye image signal processing unit 33R and the left-eye image signal processing unit 33L perform color conversion processing appropriate for each image sensor output according to the white balance condition analyzed from the left and right composite images, and the right and left composite images are optimal. In order to maintain color reproducibility, image signal processing of the same content is performed on the output image of each image sensor.
As a result, no mismatch occurs in the white balance between the right-eye image and the left-eye image.

The AF control unit 44 analyzes the left and right composite image output from the image connecting unit 42, and controls the right eye AF lens drive unit 31R and the left eye AF lens drive unit 31L so that the left and right composite image has an optimum contrast. To focus.
As a result, the focal position of the right-eye camera 3R and the focal position of the left-eye camera 3L are the same, and there is no mismatch between the focal positions of the right-eye image and the left-eye image.
The right-eye AF lens driving unit 31R and the left-eye AF lens driving unit 31L move the lens (not shown) of each camera and change the focal position according to one output of one AF control unit 44.

The image compression unit 45 functions when storing a captured image, and the left and right connected images output from the image connecting unit are compressed into, for example, JPEG format if it is a still image, or in MPEG format if it is a moving image, for example. Is output.
Since the right and left connected images are processed as one image, the compression condition for the right eye image and the compression condition for the left eye image are the same, and there is no mismatch between the compression conditions for the right eye image and the left eye image.
The storage memory unit 46 stores the image data compressed by the image compression unit 45. The operation of this part is the same as in normal shooting except that the data amount is different.

The image decompression unit 47 functions when reproducing a stored image, receives the compressed data stored in the storage memory unit, decompresses it, and restores the image before compression.
If the compressed data is three-dimensionally photographed, an image in a state where the right-eye image and the left-eye image are connected to one sheet is restored.
For this reason, the expansion condition for the right eye image and the expansion condition for the left eye image are the same, and the expansion processing is performed at the same timing, and thus there is a mismatch between the expanded right eye image and the expanded left eye image. Absent.

The image separation unit 48 selects either the image output from the image connection unit 42 or the image output from the image expansion unit 47. If the selected image is a three-dimensional image, the image separation unit 48 is connected to one image. The right-eye image and the left-eye image are separated, and the right-eye image is output to the right-eye image display unit 6R, and the left-eye image is output to the left-eye image display unit 6L.
For example, when the left and right connected images are 1280 dots wide × 480 dots high, each of the right eye image and the left eye image is 640 dots wide × 480 dots high.
The right-eye image display unit 6R and the left-eye image display unit 6L display the right-eye image and the left-eye image output from the image separation unit 48 on the display surface of the display unit (stereoscopic display panel) 6.
Since the right-eye image and the left-eye image are originally stereoscopic images using parallax, a stereoscopically viewable image is displayed on the display surface.

During stereoscopic shooting, the AF control unit 44 independently recognizes the right-eye image and the left-eye image from the left and right connected images, and the parallax between the right-eye camera 3R and the left-eye camera 3L. The AF control can be performed at higher speed by calculating the distance to the subject based on the principle of triangulation from the relationship with the position where the subject being photographed is reflected.
Further, even during moving image shooting, it is possible to adjust the focal position following the movement of the subject without changing the focal position.
In addition, stereo sound can be recorded during stereoscopic video shooting using the right and left microphones arranged in the vicinity of the camera and facing the same direction as the camera.
Further, when reproducing a stereoscopic video with stereo sound added, stereo reproduction can be performed using the right speaker and the left speaker arranged on both sides of the display unit.
Stereo audio recording / reproduction is equivalent to existing video cameras, stereo IC recorders, and the like, and a method for adding / separating stereo audio to / from MPEG images has been established.

Here, the main configuration / operation characteristics of the mobile phone with a stereoscopic photographing function according to the present embodiment will be described together.
The mobile phone with a stereoscopic photographing function according to this embodiment is a slide-type mobile terminal (mobile phone) equipped with two cameras of the same specification and a stereoscopic display panel. The other is arranged in the battery-side casing, and the terminal can be slid to obtain a parallax larger than the dimension in the terminal longitudinal direction.
The camera arranged in the display-side casing is mechanically 180 so that it can switch between photographing on the front side of the terminal (that is, the side where the photographer is located) and photographing on the back side of the terminal (that is, on the front side of the photographer). It has a structure that rotates in degrees.

The following camera shooting modes are provided.
* Planar shooting of the front side of the terminal with a camera placed in the display-side casing (first casing) in front shooting mode (self-portrait mode)
* Planar shooting of the back side of the terminal using a camera placed in the display-side casing (first casing) in the rear shooting mode (digital camera mode)
* Stereo shooting (stereo shooting) with the camera placed in the display-side housing (first housing) in the rear shooting state and the camera placed in the battery-side housing (second housing) in the rear shooting state mode)
* Stereo shooting (stereo shooting) with the camera placed in the display-side housing (first housing) in the front shooting state and the camera placed in the battery-side housing (second housing) in the front shooting state mode)

In the stereoscopic shooting mode, the right eye image and the left are controlled by controlling the shooting conditions for the two cameras (camera exposure timing control, brightness control, exposure feeling control, white balance control, focus position control). Provided with a camera control function for controlling so as not to cause a difference in photographing conditions with the eye image.
A function of linking two image signals output from two cameras to one image signal so that a right-eye image and a left-eye image can be simultaneously compressed by one-line moving image compression processing.
An image information dividing function for extracting a right-eye image and a left-eye image from two systems of connected stereoscopic image information reproduced by moving image extension processing and dividing the image into two independent image signals is provided.
A right-eye image and a left-eye image output by two cameras or output by the image information dividing function are displayed on a single display panel at the same time.

Because it is configured as described above, in the self-shooting mode and digital camera mode, while realizing the same function as a conventional camera-equipped mobile phone, in the stereoscopic shooting mode, the stereoscopic display panel is used as a viewfinder to check the stereoscopic image while taking a stereoscopic image. Can be done.
In addition, it is possible to view a stereoscopically captured image by storing the stereoscopically captured image in a memory and reproducing it.
When stereoscopic imaging is performed, the right-eye image and left-eye image capturing conditions are controlled to be the same, and it is possible to record a stereoscopic image without a sense of incongruity.
Also, a special moving image compression / decompression process for a stereoscopic image is not required, and a stereoscopic image can be compressed / decompressed by a general non-stereoscopic moving image compression / decompression process. Since there is no time lag, it is possible to record and reproduce a stereoscopic image without a sense of incongruity.

Also, in the digital camera mode and stereoscopic shooting mode, the autofocus control is much faster than the conventional TTL distance measurement method using a single camera by measuring the distance to the subject using the parallax of the two cameras. Can be done.
In addition, by attaching microphones in the vicinity of the two cameras and attaching speakers on both sides of the stereoscopic display LCD, it is possible to record a stereo sound when shooting a stereoscopic image and to play a stereo recorded sound when reproducing a stereoscopic image. It is.

As described above, the mobile phone with a stereoscopic photographing function according to the present embodiment is slidable between the first housing 1 in which the first camera 3L and the display unit 6 are disposed, and the first housing 1. A mobile phone with a three-dimensional imaging function that is placed and configured with a second housing 2 on which a second camera 3R is arranged, wherein the first camera 3L is a sliding direction of the first housing 1 The second camera 3R is arranged at the end of the second housing 2 in the direction opposite to the sliding direction of the first camera 3L, and slides the first housing 1 If it does, it will be comprised so that the distance between the 1st camera 3L and the 2nd camera 3R may become large.
Therefore, by making the first casing 1 thrust, it is possible to increase the parallax between the first camera 3L and the second camera 3R, and it is easy to capture an image suitable for generating a stereoscopic image. Can be done.

In addition, since the first camera 3L of the mobile phone with a stereoscopic photographing function according to the present embodiment is disposed so as to be capable of photographing in the same direction as the direction photographed by the second camera 3R, Normally, the first camera used in the self-portrait mode is rotated to enable stereoscopic shooting in the digital camera mode.
In addition, since the second camera 3R of the mobile phone with a stereoscopic photographing function according to the present embodiment is disposed so as to be capable of photographing in the same direction as the direction photographed by the first camera 3L, Usually, the second camera 3R used in the digital camera mode is rotated to enable stereoscopic shooting in the self-portrait mode.

In addition, since the first camera 3L and the second camera 3R of the mobile phone with a stereoscopic photographing function according to the present embodiment are arranged so as to be able to photograph both the front side and the rear side, Stereo shooting is possible in any shooting mode of the self-portrait mode and the digital camera mode.
Further, in the mobile phone with a stereoscopic photographing function according to the present embodiment, the first microphone is disposed in the vicinity of the first camera 3L, and the second microphone is disposed in the vicinity of the second camera 3R. Stereo recording is possible simultaneously with stereoscopic shooting.

Further, in the mobile phone with a stereoscopic photographing function according to the present embodiment, the first microphone rotates in conjunction with the first camera 3L, and the second microphone rotates in conjunction with the second camera 3R. Since it is configured, the first and second microphones can collect the shooting directions of the first and second cameras, respectively, and stereo recording can be performed simultaneously with stereoscopic shooting in various shooting modes. Become.
Further, in the mobile phone with a stereoscopic shooting function according to the present embodiment, the first camera 3L and the second camera 3R that operate in synchronization are controlled under the same shooting conditions, so that there is no sense of incongruity and higher quality. 3D image shooting is possible.

Further, in the mobile phone with a stereoscopic photographing function according to the present embodiment, two systems of image signals output from the first camera 3L and the second camera 3R that operate in synchronism are connected for each frame, and one system image is obtained. Since it is converted into a signal, it is possible to compress and decompress a stereoscopic image using image compression / decompression processing that does not support stereoscopic images.
Further, in the mobile phone with a stereoscopic photographing function according to the present embodiment, the distance to the subject is estimated based on the difference in the subject position in the photographed images of the first camera and the second camera that operate in synchronization. Since the focus position is automatically adjusted, the convergence of the autofocus can be accelerated in both cases of stereoscopic shooting and non-stereoscopic shooting.

  The present invention can increase the parallax between cameras, and is useful for realizing a slide-type mobile phone with a stereoscopic shooting function suitable for generating a stereoscopic image.

1 is a perspective view showing an external configuration of a mobile phone with a stereoscopic photographing function according to Embodiment 1. FIG. 3 is a block diagram illustrating a configuration of an image signal processing system in a mobile phone with a stereoscopic photographing function according to Embodiment 1. FIG. It is an external view which shows the structure of the mobile telephone which has the conventional camera imaging | photography function described in the nonpatent literature 1. It is a block diagram which shows the structure of the image processing system in the conventional mobile telephone with a three-dimensional imaging | photography reproduction function.

Explanation of symbols

1 First case (display side case)
2 Second housing (battery side housing)
3L left-eye camera (first camera)
31L Left-eye AF lens driving unit 32L Left-eye image sensor 33L Left-eye image signal processing unit 3R Right-eye camera (second camera)
31R AF lens drive unit for right eye 32R Image sensor for right eye 33R Image signal processing unit for right eye 4L Sound collection hole of left microphone (first microphone) 4R Sound collection hole of right microphone (second microphone) 5L Left Speaker sound emission hole 5R Right speaker sound emission hole 6 Display (3D display panel)
6L Left-eye image display unit 6R Right-eye image display unit 7 Operation keys arranged in the first casing (display-side casing) 8 Operation keys arranged in the second casing (battery-side casing) Reference Signs List 41 Image sensor control unit 42 Image connection unit 43 Imaging condition processing unit 44 AF control unit 45 Image compression unit 46 Storage memory unit 47 Image expansion unit 48 Image separation unit

Claims (9)

A first housing in which the first camera and the display unit are arranged, and a second housing in which the first housing is slidably mounted and the second camera is arranged. A mobile phone with a stereoscopic shooting function,
The first camera is disposed at an end of the first casing in the sliding direction,
The second camera is disposed at an end of the second casing in a direction opposite to the sliding direction of the first camera,
A mobile phone with a three-dimensional imaging function, wherein the distance between the first camera and the second camera increases when the first casing is slid.   The mobile phone with a stereoscopic photographing function according to claim 1, wherein the first camera is rotatably arranged so as to be capable of photographing in the same direction as the direction photographed by the second camera. Phone.   The mobile phone with a stereoscopic photographing function according to claim 1, wherein the second camera is rotatably arranged so that photographing in the same direction as the photographing direction of the first camera is possible. Phone.   The mobile phone with a stereoscopic photographing function according to claim 1, wherein the first camera and the second camera are rotatably arranged so as to be capable of photographing in both directions of the front side and the back side. Phone.   The first microphone is disposed in the vicinity of the first camera, and the second microphone is disposed in the vicinity of the second camera. Mobile phone with 3D shooting function.   6. The first microphone is configured to rotate in conjunction with the first camera, and the second microphone is configured to rotate in conjunction with the second camera. The mobile phone with the stereoscopic photographing function described.   The mobile phone with a stereoscopic photographing function according to any one of claims 1 to 4, wherein the first camera and the second camera that operate in synchronization are controlled under the same photographing condition. .   5. The system according to claim 1, wherein two image signals output from the first camera and the second camera operating in synchronization are connected for each frame and converted into one image signal. A mobile phone with a stereoscopic photographing function according to any one of the above.   The focus position is automatically adjusted by estimating a distance to a subject based on a difference between subject positions in images taken by the first camera and the second camera operating in synchronization. A mobile phone with a stereoscopic photographing function according to any one of 1 to 4.

Images