JP2013085018A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of solving a conventional problem that a large difference in image angle or a large displacement of optical axis causes common area to be smaller and the imaging angle to be narrower when using a function to blur the background of a specific object using a piece of distance information detected by a two-eye camera.SOLUTION: Distance information in an area other than common area in a picture to be taken by a two-eye camera is estimated based on a piece of distance information in a common area. As for the value of distance information, which cannot be estimated, a value, which is larger than the value of distance information in a specified area in the picture having a blurred background, is set. Thus, a wide angle picture with blurred background including the non-common area can be generated.

Description

  The present invention relates to an imaging apparatus having a function of acquiring distance information from a twin-lens camera and a function of blurring a background other than a designated subject.

  It is possible to take a 3D image by taking a left image of the left line of sight and a right image of the right line of sight with two cameras (two-lens cameras) having an optical axis separation that is about the distance between human eyes. .

  FIG. 5A shows an example of a portable terminal with a conventional 3D camera. A first camera 2 and a second camera 3 are arranged apart from each other in a housing 1 of the imaging apparatus. A display element 5 for displaying a 3D captured image is disposed on the rear surface of the housing 1. For example, the display element can perform 3D display using a parallax barrier type naked-eye 3D liquid crystal or the like.

  For example, Patent Document 1 discloses a method for detecting subject distance information from parallax information of left and right images of a 3D camera. This is a method of obtaining distance information (depth image) of each pixel from the difference between the horizontal coordinates of corresponding pixels of the first camera image and the second camera image, that is, parallax information.

  As described in Patent Document 1, when the angle of view of the first camera and the second camera is different, only distance information regarding an image with a narrower angle of view is obtained.

  Furthermore, by using the distance information of the subject, the photographed image similar to when the focal depth of the camera is reduced can be obtained by blurring the background of the portion excluding the specified subject of the photographed image, and the lens. It is possible to take a photo with emphasis on the subject without having to replace it. (See Patent Document 2 and Patent Document 3)

Japanese Patent Laid-Open No. 10-134187 Japanese Patent Laid-Open No. 06-231273 JP 09-181966 A

  In the function of extracting distance information from a 3D image by a conventional twin-lens camera composed of a first camera and a second camera and blurring the background of the subject specified by the user based on the distance information, the angle of view of the two cameras If the optical axes of the two cameras are different or the optical axes of the two cameras are deviated, the common area of the two camera's angles of view becomes smaller, resulting in a background blurred image with a narrower angle of view than a monocular 2D image. There was a problem.

  The present invention has been made to solve the above-described problems. Even when the angles of view of the two cameras are greatly different or when the optical axes of the two cameras are deviated, the wide range of the two cameras is improved. An object of the present invention is to realize an imaging apparatus that can obtain a background-blurred image by using a camera image of an angle of view.

  Next, means for solving the above problems will be described.

  The imaging apparatus according to the present invention includes a first camera, a second camera, distance information of each pixel of the first camera image from parallax information of the first camera image by the first camera and the second camera image by the second camera. Distance information estimating means for estimating a value and blur that blurs an area other than a designated area having a distance information value smaller than or near the first distance information value of a designated portion of the first camera image A second distance information value of a common area pixel of a common area of the first camera image and the second camera image is estimated from the parallax information, and a non-outside of the common area of the first camera The third distance information value of the non-common area pixel of the common area is estimated from the parallax information of the common area, and the blurring unit is configured to estimate the non-estimated area of the non-common area pixel that cannot be estimated from the parallax information. Estimated area image But when not the designated portion, blurring the non estimation region.

  With this configuration, by estimating distance information other than the common area of the two camera images, the two cameras can be used even when the angles of view of the two cameras are greatly different or the optical axes of the two cameras are shifted. A blurred background image using a wide-angle camera image is obtained.

  In the imaging apparatus of the present invention, the fourth distance information value of the non-estimated region is set to a distance information value larger than the first distance information value.

  With this configuration, it is possible to estimate the distance information of the portion where the distance outside the common area of the two cameras could not be estimated, and a background blurred image using a wide-angle camera image can be obtained from this distance information.

  In the imaging device of the present invention, a reference area in the common area is set in the vicinity of the first pixel in the non-common area, and the color and brightness of the reference area are equal to or less than a predetermined value in the reference area. The distance information value of the first pixel is equal to the distance information value of the second pixel when the second pixel exists, and when the second pixel does not exist, the distance of the first pixel Let the information value be the fourth distance information value.

  With this configuration, the number of pixels that can be accurately estimated even outside the common area of the two cameras increases, and the boundary line between the common area and the non-common area becomes inconspicuous in the background blurred image.

  According to the present invention, even when the angles of view of the two cameras are greatly different or when the optical axes of the two cameras are shifted, the background blurring function using a wide-angle camera image of the two cameras is provided. realizable.

1 is a schematic structural diagram and a block configuration diagram of an imaging apparatus according to a first embodiment of the present invention. The figure which shows the background blurred image generation method of Embodiment 1 of this invention The figure which shows the distance estimation method of Embodiment 1 of this invention. The flowchart of the background blurred image generation method of Embodiment 1 of this invention The figure which shows the generation method of the conventional background blur image

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a schematic structural diagram and a block configuration diagram of an imaging apparatus according to the present embodiment.

  Hereinafter, the first embodiment will be described assuming a portable terminal device including a single case. However, in the case where the portable terminal device is mounted on an electronic apparatus having a small camera such as a folding portable terminal or a digital still camera. Is the same.

  As shown in FIG. 1A, a two-lens 3D camera 4 including a first camera 2 and a second camera 3 is disposed on the back surface of the display element 5 of the rectangular housing 1 of the mobile terminal device. Here, the left camera is assumed to be the first camera, and the right camera is assumed to be the second camera.

  FIG. 1 (3) is a block diagram of the mobile terminal device. The first camera 2 and the second camera 3 are controlled by the camera control unit 6, store the captured image in the storage unit 7, and display the captured image using the display element 5.

  The display element 5 is, for example, a naked-eye 3D liquid crystal having a barrier liquid crystal, but may be a conventional liquid crystal element that cannot perform 3D display.

  The distance information estimation unit 8 has a function of detecting distance information from the parallax information of the subject from the images of the first camera 2 and the second camera 3 and generating a distance information map. Here, the distance information map refers to a map in which a one-dimensional amount including the distance between the camera and the subject or the distance information between the subjects is defined for each pixel of the captured image.

  The blur means 9 has a function of blurring the background other than the subject designated by the user, for example. Pixels that are equal to or near the distance information value of the designated portion designated by the user are extracted as designated areas, and pixels having other distance information values are blurred. As a result, a photographed image similar to that obtained when the depth of focus of the camera is reduced can be obtained, and a photograph can be taken with the subject emphasized without changing the lens.

  Note that a selection method may be used in which a pixel having a distance information value smaller than the distance information value of the designated portion is designated as a designated region without blurring.

  FIG. 1B shows the angle of view of the first camera 2 and the angle of view of the second camera 3. Although it is assumed that the angle of view of the first camera is wider than the angle of view of the second camera, it may be assumed that the angle of view of the second camera is wider than the angle of view of the first camera.

  FIG. 5B shows an example of a conventional background blurred image. Distance information is extracted from the parallax of the pixels in the common area from the first camera image 10 and the second camera image 11, and a conventional background blurred image 23 is obtained based on the distance information. The conventional background blurred image 23 is an image in which the background is blurred except for pixels having the same distance information as the person of the designated portion 18 in the common area.

  When the angle of view of the second camera is narrow, there is a problem that the angle of view of the background blurred image is very narrow compared to the first camera image.

  The above is a problem when the angle of view of the first camera and the second camera are different, but even when the angle of view of the two cameras is the same, if the optical axis direction of the two cameras is deviated due to the effects of camera mounting deviation, There arises a problem that the common angle of view of the two cameras becomes narrow.

  In the embodiment of the present invention, an implementation method will be described as a method for generating a background blurred image while maintaining the angle of view of the first camera image having a wide angle of view.

  Here, a background blurred image of the first camera image with a wide angle of view is assumed. However, if the image has an angle of view wider than the common area, it may be a part of the first camera image or the image of the second camera. It doesn't matter.

  FIG. 2 shows a background blurred image generation method according to the first embodiment.

  As shown in FIG. 2 (1), a distance information map 12 is obtained from the parallax information of the common area portion from the first camera image 10 and the second camera image 11 using the distance information estimation means 8.

  Since the angle of view of the first camera is wider than that of the second camera, the angle of view of the obtained distance information map 12 is narrower than that of the first camera image.

  The distance information map is a monochrome image that is obtained by replacing the subject distance with the luminance value of the image, and the closer to the pixel, the more white the pixel, and the farther the pixel, the black.

  As shown in FIG. 2 (2), the first camera field angle 13 consists of a common area 14 and a non-common area 15 outside of the distance information map 12. The distance information value of a part of pixels in the non-common area 15 is estimated from the disparity information of the common area 14, and as a result, the range of the estimated distance information map is expanded as shown in FIG. A method of estimating the distance information value of a part of the pixels in the non-common area 15 will be described in detail with reference to FIG.

  Next, as shown in FIG. 2 (3), for the pixel whose distance information value could not be estimated in the non-common area 15, that is, the non-estimated area pixel 22, the designated portion 18 that is not blurred in the background blurred image generation. The distance information value is greater than the distance information value, that is, the distance information value that means a distance farther than the designated portion 18. This distance information value may be a value at infinity. Even if the value is not infinitely far, it can be blurred by blurring if it is larger than the designated portion 18.

  The designated portion 18 not to be blurred is set by the user, for example.

  Thereby, the first camera distance information map 16 which is a distance information map with a wide angle of view of the first camera is completed.

  Next, as shown in FIG. 2 (4), using the first camera distance information map 16, the subject (“person” in FIG. 2 (4)) at a position substantially equal to the distance of the designated portion 18 is excluded. Thus, the first camera background blurred image 17 subjected to the blurring process is obtained.

  Next, a method for estimating the disparity information of the common area, that is, the distance information of the non-common area from the distance information value will be described in detail with reference to FIG.

  As shown in FIG. 3A, the reference pixel 20 having a color and brightness close to the non-common area pixel 19 in the non-common area 15 is searched from the common area, and when the reference pixel is found, The distance information value is set as the distance information value of the non-common area pixel 19.

  For example, when the color of the non-common area pixel 19 matches that of the reference pixel 20 with a difference equal to or less than a predetermined value, and the luminance of the non-common area pixel 19 matches with that of the reference pixel 20 with a difference equal to or less than a predetermined value, The distance information value of the reference pixel 20 is set as the distance information value of the non-common area pixel 19.

  For example, as shown in FIG. 3C, the reference pixel search range is limited to a portion close to the periphery of the common region.

  When the point closest to the boundary between the common area and the non-common area from the non-common area pixel 19 is a point P, and the distance between the non-common area pixel 19 and the point P is A1, the point P as shown in FIG. A hatched portion consisting of the length of right and left A1 and the length of upper and lower A2 is set as a reference area 21 for searching.

  By limiting the search range to the boundary portion between the common area and the non-common area, it becomes easier to estimate the correspondence between the common area and the non-common area with respect to the same subject separated by the boundary line. I do not care.

  In addition, when there are a plurality of reference pixels that are close in color and luminance to the non-common area pixel 19, for example, the distance information value is an average of the plurality of reference pixels or the reference pixel closest to the boundary line The distance information value may be adopted.

  FIG. 3B shows a distance information map after estimating distance information values of some pixels in the non-common area.

  FIG. 4 shows a flowchart of the background blurred image generation method.

  After shooting with the binocular camera (S1), distance information estimation (S2) of the common area is performed from the parallax information of the binocular camera to obtain the second distance information value of the common area.

  Next, in order to create a background blurred image, the user designates a subject (designated portion) that is not blurred (S3). The distance information value of the designated part is set as the first distance information value.

  Next, one pixel in the non-common area of the first camera with a wide angle of view is selected (S4), and a search is made as to whether or not there are reference pixels having similar colors and brightness in the common area (S5). When there is a reference pixel, the distance information value (the third distance information value) of the non-common area pixel is determined to be the same as the reference pixel (S6). A non-estimated area pixel is set, and the distance information value of the non-estimated area pixel is set to a value larger than the first distance information value of the specified portion (S7).

  The processes in S6 and S7 are performed on all the pixels in the non-common area (S8). After the process is completed, a background blurred image is generated using the distance information value to blur other than the designated part (S9).

  As described above, even when the angles of view of the two cameras are greatly different or when the optical axes of the two cameras are deviated, a background blur function using a wide-angle camera image of the two cameras can be realized.

  The present invention is useful as a 3D camera, and can be used for various electronic devices having a camera such as a mobile phone, a mobile terminal, a digital still camera, and the like.

DESCRIPTION OF SYMBOLS 1 Case 2 1st camera 3 2nd camera 4 3D camera 5 Display element 6 Camera control means 7 Memory | storage means 8 Distance information estimation means 9 Blur means 10 1st camera image 11 2nd camera image 12 Distance information map 13 1st camera Angle of view 14 Common area 15 Non-common area 16 First camera distance information map 17 First camera background blurred image 18 Designated portion 19 Non-common area pixel 20 Reference pixel 21 Reference area 22 Non-estimated area pixel 23 Conventional background blurred image

Claims (4)

Distance information estimation for estimating distance information value of each pixel of the first camera image from parallax information of the first camera and the second camera, the first camera image by the first camera and the second camera image by the second camera Means,
Blurring means for blurring a region other than the designated region having a distance information value smaller than the first distance information value of the designated portion of the first camera image or in the vicinity of the first distance information value;
A second distance information value of a common area pixel of a common area of the first camera image and the second camera image is estimated from the parallax information,
Estimating a third distance information value of a non-common area pixel of a non-common area outside the common area of the first camera from the parallax information of the common area;
The blurring unit blurs the non-estimated region when a non-estimated region pixel of the non-estimated region that cannot be estimated from the disparity information among the non-common region pixels is not the designated portion. The imaging device according to claim 1, wherein a fourth distance information value of the non-estimated region pixel is a distance information value larger than the first distance information value. The imaging apparatus according to claim 1, wherein the fourth distance information value is a distance information value at infinity. A reference region in the common region is set in the vicinity of the first pixel in the non-common region, and the second pixel has a color and luminance that are different from the first pixel by a predetermined value or less in the reference region. When present, the distance information value of the first pixel is set equal to the distance information value of the second pixel, and when the second pixel does not exist, the distance information value of the first pixel is set to the fourth distance information value. The imaging apparatus according to any one of claims 1 to 3.

Images