JP2002300454A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device which obtain images, in which parallax reduced in self portrait photographing or the like. SOLUTION: The imaging device is equipped with a couple of cameras 3, 5, a display 6, and an image processor which processes images photographed with respective cameras. The light-receiving surfaces of lenses 3a, 5a of the cameras 3, 5 are positioned on the display surface side of the display 6. Viewed from display surface side, the respective lenses are arranged in a row to sandwich the display 6. The image processor determines the center line of each image, in the direction vertical to the direction where the respective lenses are arranged, and obtains an image on the side nearer than central line, regarding the respective images. The obtained images are so composited that the center lines of the obtained images coincide with each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus having a camera, such as a digital camera, and a display unit, for example, capable of self-portrait photographing.

[0002]

2. Description of the Related Art Generally, a portable telephone has a display unit such as a liquid crystal panel. Recently, a mobile phone equipped with a digital camera has been widely used. FIG. 9 shows such a mobile phone 200.
A display unit 60 such as a liquid crystal panel and a digital camera 30 are disposed on one surface side of the mobile phone 200, and a lens 30 a of the digital camera 30 is separated to the upper right of the display unit 60. Are located. In the case of a mobile phone with a built-in digital camera, so-called self portrait photography (self photography) is possible.

In self-portrait photographing, first, a digital camera is pointed at a subject (user's face), and a captured image is displayed on a display unit to monitor the appearance of the face. Next, the user saves the captured image in the memory by performing a save operation when he / she likes the image while looking at the image of the face on the display unit.

In the case of self-portrait photographing, the user's line of sight is not the camera but the direction of the display unit. This is called parallax (parallax) that occurs between an angle when the subject is viewed from the camera lens and an angle when viewed from the display unit. This parallax will be described with reference to FIGS.

FIG. 10 is a diagram showing the positional relationship between the face (eyes) of the user (subject) displayed on the display unit 60 and the lens 30a, and FIG. 11 is an overhead view of FIG.

In FIG. 10, the distance between the right eye of the user staring at the display unit 60 and the lens 30a is 45 mm,
The distance between the left eye and the lens 30a is 40 mm. In FIG. 11, the distance between the user's face and the lens 30a is 2
00 mm. Then, the angle (parallax) θ between the actual line of sight of the right eye staring at the display unit 60 and the direction in which the right eye image is incident on the lens 30a is about 12.7 °. Similarly, the angle between the left eye line and the direction in which the left eye image enters the lens 30a is about 11.3 °. These parallaxes are so large that they cannot be ignored, and there is a problem that the above-mentioned image having an unnatural eye line is obtained.

For this reason, various measures have been taken, such as installing a camera lens at the center of the display unit or installing a camera behind the display unit as a half mirror.

[0008]

However, when a lens is installed at the center of the display unit, there is a problem that the center of the display unit is a non-display area. In addition, when a half mirror is used, the entire device including the camera becomes large, so that there is a problem that it is not suitable for an imaging device, particularly a device such as a portable terminal that requires small size and light weight.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned problems in an imaging apparatus having a camera and a display unit, and to provide an imaging apparatus capable of obtaining an image with reduced parallax in self-portrait photography or the like.

[0010]

According to a first aspect of the present invention, there is provided an imaging apparatus comprising: a set of cameras;
A display unit, and an image processing unit that processes images captured by the pair of cameras, wherein a light receiving surface of a lens of the camera is located on a display surface side of the display unit, and the display surface is When viewed from the side, the lenses are arranged in a row so as to sandwich the display unit, and the image processing unit determines a center line of the images in a direction perpendicular to a direction in which the lenses are lined up. In the image, images closer to the lens than the center line are obtained, and the obtained images are combined so that the center lines of the obtained images coincide with each other.
With this configuration, when capturing an image of a subject facing the display unit, on the side closer to the lens than the center line of the subject,
Parallax between the corner of the subject viewed from the lens and the corner of the display viewed from the display unit is reduced. Then, the image processing unit combines the images closer to the lens than the center line among the captured images, that is, the images with reduced parallax,
Parallax of the combined image is reduced as a whole.

According to a second aspect of the present invention, the image pickup apparatus further comprises control means for displaying the composite image on the display unit. In this case, the user (subject)
It is possible to confirm its own imaging state.

According to a third aspect of the present invention, in the image pickup apparatus, each of the lenses is arranged in the left-right direction of the display unit. In this way, the parallax of the subject in the left-right direction can be effectively reduced, so that a natural image in which the eyes of the face face the lens can be obtained.

According to a fourth aspect of the present invention, the respective lenses are arranged adjacent to an outer peripheral edge of the display section. In this case, since the lens is closer to the subject, parallax is further reduced.

According to a fifth aspect of the present invention, in the image pickup apparatus, the image processing unit is arranged to divide each of the images by a predetermined width in a direction in which the lenses are arranged, and to divide the image in a direction perpendicular to the direction in which the lenses are arranged. The method is characterized in that a change in luminance in a direction is acquired, and when the change in luminance coincides between regions belonging to each image, the region is set as the center line of the corresponding image. By doing so, the center line of the image can be accurately determined,
Problems such as distortion of images at the time of image synthesis are less likely to occur.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram illustrating an appearance of an embodiment of an imaging device (mobile phone) 20 according to the present invention. The mobile phone 20 includes a vertically long rectangular display unit 6 including a liquid crystal panel, a digital camera (camera) 3,
5, an extendable wireless antenna 10a, an input key 14 for performing an input operation, and the like. The digital cameras 3 and 5 include lenses 3a and 5a, respectively, and the light receiving surfaces of the lenses 3a and 5a and the input keys 14 are located on the display surface side of the display unit 6 (paper side in the drawing). Similarly, when viewed from the display surface side, the lenses 3a, 5a are arranged in a line in the left-right direction with the display unit 6 interposed therebetween, and are arranged adjacent to the vertical sides of the display unit 6, respectively. Each of the lenses 3a and 5a is located slightly above the center of the vertical side of the display unit 6. In addition,
In this embodiment, the imaging device is a mobile phone or a PHS (Pers
An example is the case of an onal handyphone system (PDA), but the present invention is not limited to this, and PDA (Personal Digital Assistants:
The present invention is also applicable to personal information devices, portable personal computers, video cameras, and the like. or,
The “left-right direction” refers to a direction parallel to the direction of the image displayed on the display unit 6.

FIG. 2 is a block diagram showing the configuration of the mobile phone 20. The mobile phone 20 includes a control unit 2 for controlling the whole,
Digital cameras 3 and 5, an image processing unit 4 for processing an image captured by the digital camera, the display unit 6, a storage unit (memory) 8 for storing various data, a wireless antenna 10a,
The system includes a wireless transmission / reception unit 10 for transmitting / receiving a signal via a wireless antenna 10a, a signal processing unit 12 for processing the signal, an input key 14, a codec 16, a speaker 16a for inputting / outputting an audio signal, and a microphone 16b.

Here, the antenna 10a transmits and receives radio signals to and from a base station. The control unit 2 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Rand Rand).
om Access Memory). The CPU performs arithmetic processing according to a predetermined control program. The ROM stores the control program and control data, and the RAM temporarily reads and writes various data necessary for the arithmetic processing. The storage unit (memory) 8 stores image data that is appropriately captured.

Next, the positional relationship between the display unit, the lens, and the subject (face) when performing self-portrait photographing with the mobile phone 20 will be described with reference to FIGS. FIG. 3 shows the user's face displayed on the display unit 6,
FIG. 4 is a diagram illustrating a positional relationship between the lenses 3a and 5a and the display unit 6, and FIG. 4 is an overhead view of FIG.

In FIG. 3, images picked up by the digital cameras 3 and 5 are displayed on the display unit 6 through image processing which will be described later. When the is pressed, the captured image is stored in the memory 8. Then, the user's right eye displayed on the display unit 6 and (right side)
The distance from the lens 5a is 20 mm, and the distance between the user's left eye and the (left) lens 3a is 20 mm. In the present invention, “imaging” is not limited to actually capturing (saving) an image, but includes an image formed by the lenses 3a and 5a.

In FIG. 4, which is a bird's-eye view of FIG. 3, the distance between the user's face and each of the lenses 3a, 5a is 200.
mm. Then, when the right eye is staring at the display unit 6, the actual line of sight of the right eye (the corner when the right eye is viewed from the display unit)
(Parallax) φ between the right-eye image and the direction in which the right-eye image is incident on the right lens 5a (the angle when the right eye is viewed from the lens)
2 is about 5.7 °. Similarly, the angle φ1 between the left eye line and the direction in which the left eye image is incident on the left lens 3a is also about 5.7 °. This value is low for parallax and does not result in an unnatural image.

On the other hand, compared to the right eye, the left eye is the right lens 5a.
, The angle γ between the actual line of sight of the left eye and the direction in which the left eye image is incident on the right lens 5a.
Is larger than the angle φ2. With the actual eyes of the right eye,
An angle (not shown) between the right-eye image and the direction in which the right-eye image is incident on the left lens 3a has a similar relationship.

FIG. 5 shows an image formed by each lens. First, in the image formed by the right lens 5a (FIG. 5A), the line of sight of the right eye is substantially directed to the lens direction due to the size of the angle φ2 and the angle γ, but the line of sight of the left eye is located outside the lens direction. It turns out that it is suitable for. The same applies to the image formed by the left lens 3a (see FIG. 5).
(2)) Conversely, the line of sight of the left eye points substantially toward the lens direction, but the line of sight of the right eye points outward from the lens direction. In other words, in each image, a natural eye line with less parallax can be obtained on the side closer to the lens (the right side of the face in FIG. 5A), but an unnatural eye line on the side farther from the lens. I understand. In the figure, the eyes are exaggerated.

As described above, of the images, the image farther from each lens has a large parallax. Therefore, the present invention removes an unnatural portion from each image and combines the remaining portions to obtain a natural image (with eyes and the like) as a whole. Such image processing will be described with reference to FIGS.

FIG. 6 shows a flow of processing performed by the image processing section 4. In this figure, the image processing unit 4 determines the center line of each image after capturing each image captured by the digital cameras 3 and 5 as data (step S100).
Here, the center line indicates the center of the image in a direction (vertical direction) perpendicular to the direction in which the lenses 3a, 5a are arranged (left-right direction in the figure). The method of determining the center line will be described later. Next, the image processing unit 4 recognizes an image area of each image closer to the lens than the center line, and acquires data of the image (Step S102). Here, “the side closer to the lens” means, for example, the image on the left side of the center line in the case of FIG. Then, the image processing unit 4 combines the images so that the center lines of the acquired images match (step S104).

FIG. 7 is a schematic diagram showing a transition of an image by the processing of FIG. First, the center lines of the images formed by the right lens 5a and the left lens 3a are determined (FIG. 7 (1)). Next, of the images formed by the right lens 5a, an image on the left side of the center line and an image formed by the left lens 3a on the right side of the center line, that is, images on the sides closer to the lens are obtained. (Figure 7
(2)). Then, the images are synthesized in a state where the center lines of the obtained images are aligned (FIG. 7 (3)). In this manner, by removing the large parallax image farther from each lens from the images captured by each digital camera and synthesizing the small parallax image among the images, the parallax as a whole is reduced. Is reduced,
Natural images can be obtained.

The control unit 2 appropriately displays the composite image on the display unit 6 or saves the composite image in the storage unit 8. In a later-described video conference or the like, the composite image is transmitted to the destination terminal.

Next, an example of a method for determining the center line will be described with reference to FIG. In this figure, the image processing unit 4
8 divides an original image into a plurality of regions with a width w in a direction (lateral direction) in which the lenses 3a and 5a are arranged (see FIG. 8).
(1)). FIG. 8B shows one divided region R. Then, for each area, a luminance change (a pattern of luminance change with respect to displacement) in a direction (vertical direction in the figure) perpendicular to the direction in which the lenses are arranged is measured and acquired (FIG. 8 (3)).
Here, by scanning the region R from top to bottom in FIG. 8 (2), a brightness change in which the hair, eyebrows, eye line, and lip outline portions become shadows (low brightness) in order is obtained. Become.

The image processing section 4 performs the above-described area division and the measurement of the luminance pattern for each image captured by the left and right digital cameras, and when the luminance patterns match with each other, the area is determined. The center line of the corresponding image. For example, when the luminance pattern of the region R in FIG. 8 matches the luminance pattern of the region S (not shown) belonging to another image, the region R is set as the center line of the image in FIG. Center line of the image.

The present invention is not limited to the above embodiment. For example, the lenses 3a, 5a may be arranged in a line in the vertical direction in FIG. With this configuration, among the images formed by the upper and lower lenses, the image farther from each lens is removed, so that the parallax of the subject in the vertical direction can be reduced.

In the above-described embodiment, the self-portrait photographing (self-photographing) for displaying the composite image of the image photographed by each camera on its own photographing apparatus has been described. The parallax can be similarly reduced when the composite image is displayed on the display unit of another imaging device (mobile phone) as in the case of performing the above.

[0031]

As described above, according to the present invention,
When capturing an image of a subject facing the display unit, on the side closer to the lens than the center line of the subject, parallax (parallax) between the angle viewed from the lens of the subject and the angle viewed from the display unit is reduced. Then, the image processing unit combines the images closer to the lens than the center line of the captured images, that is, the images with reduced parallax, so that the combined image can reduce parallax as a whole.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an appearance of an imaging device (mobile phone) of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a mobile phone.

FIG. 3 is a diagram illustrating a positional relationship among a subject, a lens, and a display unit.

FIG. 4 is an overhead view of FIG. 3;

FIG. 5 is a diagram showing an image formed by each lens.

FIG. 6 is a diagram showing a processing flow performed by an image processing unit 4.

FIG. 7 is a schematic diagram showing a transition of an image according to the processing flow of FIG. 6;

FIG. 8 is a diagram showing a procedure for determining a center line.

FIG. 9 is a diagram showing the appearance of a conventional imaging device (mobile phone).

FIG. 10 is a diagram illustrating a positional relationship among a subject, a lens, and a display unit in a conventional imaging apparatus.

FIG. 11 is an overhead view of FIG. 10;

[Explanation of symbols]

 2 control unit 3, 5 (digital) camera 3a, 5a lens 4 image processing unit 6 display unit 20 imaging device (mobile phone)

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2H054 AA01 5C022 AA12 AB61 AB62 AB68 AC03 AC42 AC69 AC77 AC78 5C064 AA01 AA04 AB03 AC02 AC08 AC12 AD02 AD08 AD14 5K027 AA11 BB01 FF01 FF22 HH29

Claims (5)

[Claims] 1. A camera comprising: a set of cameras; a display unit; and an image processing unit configured to process images picked up by the set of cameras. Located on a display surface side, and when viewed from the display surface side, each lens is arranged in a row so as to sandwich the display unit, and the image processing unit is arranged in a direction perpendicular to a direction in which the lenses are arranged. The center line of each image is determined, an image closer to the lens than the center line is obtained in each image, and the obtained images are combined so that the center lines of the obtained images match. An imaging device, comprising: 2. The imaging apparatus according to claim 1, further comprising control means for displaying the composite image on the display unit. 3. The imaging device according to claim 1, wherein each of the lenses is disposed in a left-right direction of the display unit. 4. The imaging apparatus according to claim 1, wherein each of the lenses is disposed adjacent to an outer peripheral edge of the display unit. 5. The image processing unit obtains a luminance change in a direction perpendicular to the direction in which the lenses are arranged, for each of the regions obtained by dividing the images by a predetermined width in the direction in which the lenses are arranged, The imaging device according to any one of claims 1 to 4, wherein, when the change in luminance coincides between regions belonging to each image, the region is set as a center line of the corresponding image.