JP2013143741A - Image photographing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image photographing device capable of realizing a high picture quality due to high-precision pixel displacement at a small constitution.SOLUTION: An image photographing device 1 comprises: an imaging module CM having an imaging lens and a pixel-type imaging element; and a tilting device arranged outside a main body CM1 of the imaging module CM and executing tilting processing by tilting the imaging module CM against a face vertical to an optical axis by a minute amount. Also, the tilting device 22 has a drive section 23 for executing tilting processing by directly transmitting a displacement amount along an optical axis direction to an extension section CM2 of the imaging module CM. Furthermore, the drive section 23 changes a position of an imaging element relative to a subject image before and after the tilting processing in a diagonal direction by an odd number of times of a half of an array pitch that is a distance between adjacent pixels in a diagonal direction of the pixels.

Description

  The present invention relates to an image capturing apparatus, and more particularly to an image capturing apparatus capable of capturing a high-resolution image by performing pixel shifting and capturing a plurality of figures.

  With recent advances in semiconductor technology, there is an increasing need for camera miniaturization in digital cameras, mobile phone terminals, and the like. In order to achieve downsizing, the imaging unit (imaging system) needs to be configured compactly. However, since the size of the imaging unit is limited by the number of pixels of the imaging device, the number of pixels is sought when downsizing the imaging unit Is limited. Therefore, there is a problem that it is difficult to achieve both downsizing and resolution.

  On the other hand, attempts have been made to increase the number of pixels in solid-state imaging devices such as CCDs and CMOSs used for camera applications in order to increase the resolution of images. The size of each pixel is the same as in the conventional case, and if only the number of pixels is increased, the area of the solid-state imaging device is increased. Therefore, the number of elements obtained from one wafer is reduced and the yield is lowered, resulting in an increase in cost. For this reason, pixel size and pixel pitch are reduced to increase the number of pixels, but in general, sensitivity and saturation output are in proportion to the pixel size, so resolution can be achieved only by reducing the pixel size. There is a limit to improvement.

  Therefore, a method called “pixel shifting” has been proposed as a method for increasing the resolution without increasing the number of pixels. In other words, pixel shifting is a method in which a subject image is formed on a solid-state image sensor by shifting the subject image by time by half the pixel pitch, and the subject image is sampled at a higher spatial frequency than determined by the original pixel pitch. is there. If the subject image is shifted in one direction, an image equivalent to that captured with twice the number of pixels can be obtained. In addition, when shifted in two orthogonal directions, an image equivalent to an image captured with four times the number of pixels can be obtained.

  In contrast to pixel shifting in two orthogonal directions, a method has also been proposed in which the pixels of the solid-state imaging device are shifted in an oblique direction when the pixels of the solid-state imaging device are in a square regular array. It is known that the frequency component in two orthogonal directions is doubled by shifting the pixel by half a pixel in an oblique direction, so that a pseudo four-fold number of pixels is output by image processing. In addition, it is known that higher resolution can be achieved depending on the number of times of photographing, such as photographing three times by shifting one-third pixel obliquely instead of half a pixel.

  Various techniques have been proposed for the pixel shifting technique. For example, the technique disclosed in Patent Document 1 discloses an electronic camera that realizes a pixel shift effect by controlling and driving an optical camera shake correction mechanism mounted on a photographing lens.

  Further, the technology disclosed in Patent Document 2 discloses a solid-state imaging device that realizes a pixel shifting effect by moving a solid-state imaging device using a bimorph.

  Further, the technology disclosed in Patent Document 3 discloses an image reading apparatus that realizes a pixel shifting effect by integrally moving a CCD area sensor and an imaging lens in parallel.

  Further, the technology disclosed in Patent Document 4 discloses an imaging device that realizes a pixel shifting effect by driving a liquid crystal element.

JP 2006-345147 A Japanese Unexamined Patent Publication No. 60-18958 Japanese Patent Laid-Open No. 2005-151351 JP 2005-318511 A

  However, the technique of Patent Document 1 has a problem that since a shake driving device has a very large driving amount, it is very difficult to repeatedly drive a small amount of half-pixel shift with high accuracy. Further, when driving with a camera shake lens, there arises a problem that the amount of deviation varies depending on the uniform correction state due to the distortion aberration inherent in the camera shake correction lens.

  Moreover, in the technique of Patent Document 2, the solid-state imaging device is usually arranged on a substrate or a flexible board, and the size of the apparatus is increased to drive them together. Further, since the driving device has a thickness in the optical axis direction, there is a problem that it is difficult to realize a reduction in thickness. Further, when the image pickup device is driven, there is a problem that the image pickup device is tilted and the image quality is deteriorated such as a tilt of the image plane.

  Further, the technique of Patent Document 3 is effective only when the object distance is fixed, and can be used for the extension lens proposed in Patent Document 3, but cannot be used for a general image capturing apparatus in which the object distance varies. There is a problem.

  Furthermore, in the technique of Patent Document 4, when achieving a thin optical system, it is not preferable to insert an element in the optical system because the thickness increases. In addition, the liquid crystal element is generally not driven at high speed, and the frame interval resulting from it is greatly affected by camera shake and subject blurring. Therefore, the effect of image quality degradation such as camera shake is greater than the image restoration effect of pixel-shifted shooting. There is a problem of becoming.

  As described above, the conventional technology is not an appropriate solution to the problem of achieving both the downsizing and the resolution described above.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image photographing apparatus that has a compact configuration and achieves high-quality image output by highly accurate pixel shifting.

  In order to solve the above problems, an invention according to claim 1 is an imaging module having a main body unit including an imaging lens and a pixel-type imaging device that forms a subject image formed through the imaging lens during imaging processing. A tilting device that is disposed outside the main body of the imaging module and tilts the imaging module by a small amount with respect to a plane perpendicular to the optical axis to perform a tilting process, and the tilting device includes the light A driving unit that executes the tilting process by directly transmitting a displacement amount along the axial direction to a predetermined driven part of the imaging module, and the driving unit includes the subject before and after the tilting process. The position of the image sensor with respect to an image is changed in the predetermined direction by an odd multiple of an array pitch that is a distance between adjacent pixels in the predetermined direction of the pixel. To, is an image capturing device.

  According to a second aspect of the present invention, there is provided the image photographing device according to the first aspect, wherein the driving unit is constituted by a piezoelectric element that expands and contracts along the optical axis direction.

  The invention according to claim 3 is the image photographing device according to claim 2, wherein the pixels of the image pickup device include a plurality of pixels arranged in a regular square pattern, and the predetermined direction is the square regular array. The diagonal direction is included.

  According to a fourth aspect of the invention, there is provided the image photographing device according to the first or second aspect, wherein the drive unit is disposed in a direction perpendicular to the optical axis with respect to the imaging module. .

  The invention according to claim 5 is the image photographing device according to claim 4, wherein the imaging module has an extending portion formed to extend from a side surface of the main body portion in a direction perpendicular to the optical axis. The predetermined driven portion includes the extending portion, and the driving portion performs the tilting process by abutting from below the extending portion and moving the extending portion in the optical axis direction.

  The invention according to claim 6 is the image photographing device according to claim 2, wherein the formation length of the drive unit in the optical axis direction is greater than the formation length of the imaging module in the optical axis direction. Also features a short.

  The invention according to claim 7 is the image photographing device according to claim 1 or 2, wherein the time required for one tilting process by the driving unit is not less than 0.25 ms and not more than 5 ms. It is characterized by.

  The invention according to claim 8 is the image photographing device according to claim 1, wherein the image formed on the image sensor at the time of the imaging process is processed and a display image is output to a predetermined display unit. Image processing means for performing image processing; processing for the imaging module; tilting processing for the tilting device; and control means for controlling the image processing in the image processing means, the image processing means The first image formed on the image sensor by the photographing process before execution of the tilting process in the tilting device under the control of the control means, and the imaging process after the tilting process is performed. Using the second image formed on the image sensor, the image processing is executed to obtain the display image.

  The invention according to claim 9 is the image photographing device according to claim 8, further comprising an electronic zoom unit that cuts out a part of an image formed on the image sensor and changes a photographing magnification, The image processing means selectively executes the image processing in an area cut out by the electronic zoom means.

  According to the first to ninth aspects of the present invention, an imaging module having an imaging lens, a pixel-type imaging device that forms a subject image formed through the imaging lens during imaging processing, and a main body of the imaging module. A tilting device that is disposed outside and tilts the imaging module by a small amount with respect to a plane perpendicular to the optical axis to execute a tilting process. This eliminates the need to change the imaging module and avoids an increase in the size of the imaging module, thereby simplifying the overall configuration of the image capturing apparatus compared to a conventional apparatus that changes the pixel position. Further, since the imaging module itself may be the same as the conventional one, the image quality of the original image capturing apparatus is not deteriorated.

  In addition, the tilting device includes a drive unit that executes tilting processing by directly transmitting a displacement amount along the optical axis direction to a predetermined driven portion of the imaging module. That is, the drive amount depends only on the displacement amount obtained from the physical constant and shape of the drive element. For this reason, it is not necessary to consider the drive error, the drive accuracy is improved, and high-precision repeatability can be realized. As a result, high-quality image acquisition is possible.

  Further, the drive unit changes the position of the image sensor with respect to the subject image before and after the tilting process in a predetermined direction by an odd multiple of an array pitch that is a distance between adjacent pixels in a predetermined direction of the pixels. This makes it possible to sample the subject image at a higher spatial frequency than determined by the pixel pitch.

  As a result, it is possible to provide an image photographing apparatus that achieves both a compact configuration and high image quality output.

  According to the second aspect of the present invention, the drive unit is composed of a piezoelectric element that expands and contracts along the optical axis direction. Thereby, the processing speed of the tilting process can be increased, and the photographing time can be shortened. As a result, it is possible to reduce the influence of subject blur and camera shake.

  According to the invention of claim 3, the pixels of the image sensor include a plurality of pixels arranged in a square regular array, and the predetermined direction includes a diagonal direction of the square regular array. As a result, frequency information in two orthogonal directions in the square regular array can be doubled only by performing the tilt process for changing the position in the diagonal direction of the square regular array once. For this reason, the image quality can be improved uniformly regardless of the directionality of the subject.

  According to invention of Claim 4 or 5, the drive part is arrange | positioned with respect to an imaging module at the orthogonal | vertical direction with respect to an optical axis. Thereby, the tilting process can be performed without giving a thickness in the optical axis direction of the imaging module. As a result, it is possible to provide an image capturing apparatus that achieves both a compact configuration and high image quality output.

  According to the sixth aspect of the present invention, the formation length in the optical axis direction of the drive unit is shorter than the formation length in the optical axis direction in the main body portion of the imaging module. Thereby, even if a drive unit is newly provided, it is not necessary to give the entire apparatus a thickness in the optical axis direction of the imaging module, and a thin configuration can be realized. As a result, it is possible to provide an image capturing apparatus that is thin and has a compact configuration and high image quality output.

  According to the seventh aspect of the present invention, it is possible to minimize the influence of camera shake by reducing the time required for one tilting process by the drive unit to 5 ms or less. In addition, by setting the pixel shift drive time of the drive unit to 0.25 ms or more, it is possible to avoid the influence of damping vibration after driving and the enlargement of the tilting device.

  According to the eighth aspect of the present invention, the image processing means includes the first image formed on the image sensor by the photographing process and the after the tilting process before the tilting process in the tilting device is performed under the control of the control means. The image processing is executed using the second image formed on the image sensor by the imaging process to obtain a display image. Thereby, a high-quality display image can be obtained.

  According to the ninth aspect of the present invention, the image processing means selectively executes image processing in the area cut out by the electronic zoom means. This makes it possible to obtain an image having a resolution equivalent to that of an image captured using an optical zoom unit that optically changes the angle of view as a display image.

It is a figure which shows schematic structure and the function of the imaging device concerning 1st and 2nd embodiment. It is a schematic diagram explaining the specific structure of the image imaging device which concerns on 1st Embodiment. It is a schematic diagram explaining the specific structure of the image imaging device which concerns on 1st Embodiment. It is a schematic diagram explaining the specific structure of the image imaging device which concerns on 1st Embodiment. It is a schematic diagram explaining the image processing of diagonal pixel shift. It is a figure which illustrates that pixel shift amount changes with image heights. It is a figure which illustrates that pixel shift amount changes with image heights. It is a schematic diagram explaining the image processing of diagonal pixel shift. It is a figure which illustrates frequency distribution of hand shake angle. It is a flowchart explaining the basic operation | movement implement | achieved in the image imaging device which concerns on 1st Embodiment. It is a schematic diagram explaining the specific structure of the image imaging device which concerns on 2nd Embodiment. It is a schematic diagram explaining the image processing of XY pixel shift. It is a schematic diagram explaining the image processing of XY pixel shift. It is a flowchart explaining the basic operation | movement implement | achieved in the image imaging device which concerns on 2nd Embodiment. It is a flowchart explaining the basic operation | movement implement | achieved in the image imaging device which concerns on 2nd Embodiment. It is a figure which shows schematic structure and the function of the imaging device which concerns on 3rd Embodiment. It is a flowchart explaining the basic operation | movement implement | achieved in the image imaging device which concerns on 3rd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, parts having the same configuration and function are denoted by the same reference numerals, and redundant description is omitted in the following description. Further, the drawings are schematically shown, and the sizes, positional relationships, and the like of various structures in the drawings are not accurately illustrated. For the purpose of clarifying the azimuth relationship of each figure, the coordinate axes of the right-handed XYZ orthogonal coordinate system are attached to FIGS. 2 to 4, 11, and 12.

<1. First Embodiment>
<1-1. Overview of image capture device>
FIG. 1 is a diagram showing a schematic configuration and functions of an image capturing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the image capturing apparatus 1 includes an imaging module CM, and mainly includes a control unit 20, an operation unit 21, a tilting device 22, an image processing unit 3, and a display unit 28. ing.

  The imaging module CM includes an imaging lens (not shown) and a pixel-type imaging element SR (see FIGS. 2A and 4 to be described later) that forms a subject image formed through the imaging lens during imaging processing. The main body part CM1 and the extending part CM2 described later are included. The imaging module CM is configured to image a subject OB in front of the camera, and an image signal (hereinafter abbreviated as “image”) obtained by imaging is transmitted to the image processing unit 3 via a data line CB.

  Hereinafter, in this specification, an image acquired by the first shooting of the imaging module CM is referred to as a “first image”, and an image acquired by the second and subsequent shootings of the imaging module CM is referred to as a “second image”. Called. That is, an image captured before the pixel shifting process described later is performed is used as the first image, and an image captured after the pixel shifting process is performed is used as the second image.

  The control unit 20 comprehensively controls the imaging process for the imaging module CM, the tilting process for the tilting device 22, and the image processing in the image processing unit 3.

  The operation unit 21 includes operation buttons and the like. When the operator operates the operation unit 21, external information such as the start of a shooting operation is input to the control unit 20.

  The tilting device 22 tilts the imaging module CM with a slight amount of tilting with respect to a plane perpendicular to the optical axis under the control of the control unit 20 to execute a tilting process. Details of a specific arrangement example and configuration of the tilting device 22 will be described later.

  The display unit 28 is configured by a liquid crystal display or the like, and displays a display image that has been subjected to necessary processing by the image processing unit 3.

  The image processing unit 3 performs image processing for processing an image formed on the image sensor during the imaging process and outputting a display image to the display unit 28 (predetermined display unit). The image processing unit 3 functions as an information processing unit, and mainly includes a storage device 34, an input / output unit 35, an arithmetic control unit 36, and an interface (I / F) 37.

  The storage device 34 is composed of, for example, a memory element such as SRAM, and stores first and second images obtained by imaging by the imaging module CM.

  The input / output unit 35 includes, for example, a semiconductor memory card slot, sets a portable storage medium such as a flash memory, and exchanges data with the arithmetic control unit 36.

  The arithmetic control unit 36 includes a CPU 36a that functions as a processor and a memory 36b that temporarily stores information, and controls the respective units of the image processing unit 3 through these digital processing circuits 36a and 36b. In the arithmetic control unit 36, various functions, various information processing, and the like are realized by reading and executing the program PG in the storage device 34. The memory 36b can store program data stored in the portable storage medium via the input / output unit 35. The stored program can be appropriately reflected in the operation of the image processing unit 3.

  In addition, the arithmetic control unit 36 generates a display image by performing image processing by pixel shift described later. That is, under the control of the control unit 20, the first image formed on the image sensor by the photographing process before execution of the tilting process in the tilting device 22 and the image formed on the image sensor by the photographing process after execution of the tilt process. Using the second image thus obtained, image processing is executed to obtain a display image. Then, the obtained display image is output to the display unit 28 according to a command from the control unit 20.

  The interface (I / F) 37 is connected to the imaging module CM via the data line CB, and enables transmission and reception of data acquired by the imaging module CM.

<1-2. Tilt device>
In order to realize a thin image pickup device in the pixel shifting method, the whole image pickup module is tilted instead of the conventional method of increasing the size and thickness such as driving of the image pickup element, liquid crystal insertion, and camera shake drive. The method of letting you do is the best. Therefore, the image capturing apparatus 1 includes a tilting device 22 that is disposed outside the main body CM1 of the imaging module CM and tilts the imaging module CM with a slight amount with respect to a plane perpendicular to the optical axis to execute a tilting process.

  2 to 4 are schematic diagrams illustrating specific configurations of the tilting device 22 and the imaging module CM of the image capturing device 1. Here, a description will be given assuming a case of oblique pixel shift described later.

  As shown in FIG. 2A, when the XY plane of the image capturing apparatus 1 is viewed from the + Z direction with the opening 25 in the main body CM1 of the imaging module CM facing upward (+ Z direction), the tilting device 22 is The imaging module CM having a substantially rectangular parallelepiped shape is disposed outside the main body CM1. Further, the imaging element SR inside the main body CM1 is arranged on the flexible substrate 26 as shown in FIG. 4A.

  Here, in the diagonal pixel shifting process for the pixels having the square regular arrangement, the position of the pixel of the image sensor SR with respect to the subject image must be moved in the arrow ARS direction which is the diagonal direction of the pixel (FIG. 4B). reference). Therefore, in the present embodiment, as the diagonal pixel shifting process, a tilting process that tilts the entire imaging module CM with respect to a plane (XY plane) perpendicular to the optical axis direction (Z direction) is employed. In order to realize this tilting process, the tilting device 22 drives the drive unit 23 in the arrow AR1 direction (see FIG. 2A) corresponding to the arrow ARS direction (see FIG. 4B) that is the diagonal direction of the square pixel. And a fulcrum part 24 are provided.

<1-2-1. Configuration of drive unit for high accuracy>
By the way, when considering high image quality output by image processing after imaging, the accuracy of pixel movement is very important. If the amount of movement deviates significantly from the desired amount of movement, erroneous information is included in an image that should have been output with high image quality by image processing, and a high-resolution high-resolution image cannot be obtained.

  For example, when a displacement enlarging mechanism using the principle of gear or lever is used, a driving error of the displacement enlarging mechanism becomes an error of pixel shift and movement amount. For this reason, the method of directly transmitting the drive of the drive unit 23 of the tilting device 22 is most preferable. When displacement is performed by direct transmission, the drive amount depends only on the displacement amount obtained from the physical constant and shape of the drive element, and there is no drive error of the displacement enlargement mechanism, so there is an advantage that repeatability is very high. .

  Therefore, in order to realize high-precision repeatability, in the image processing apparatus 1, the driving unit 23 transmits the displacement amount along the optical axis direction directly to a predetermined driven portion of the imaging module CM, thereby performing tilt processing. Execute. Specifically, as shown in FIG. 2B, the drive unit 23 is provided apart from the main body CM1 of the imaging module CM, and the main body is perpendicular to the optical axis with respect to the main body CM1. It arrange | positions so that abutting is possible for the lower surface of extending | stretching part CM2 formed extending from the side surface of CM1 in the direction perpendicular to the optical axis. Therefore, the predetermined driven portion is set as the extending portion CM2. Thereby, the drive unit 23 performs a tilting process on the entire imaging module CM by abutting from below the extending unit CM2 and moving the extending unit CM2 in the optical axis direction. However, the extending part CM2 is not limited to the configuration like the extending part CM2 shown in FIG. The extending portion has a structure provided outside the imaging effective range of the imaging element SR, and may have an action point in the optical axis direction. For example, the extending portion may have an opening shape excluding the driving unit 23. Absent. Similarly, the fulcrum portion 24 is not limited to the shape shown in FIG. 2 as long as it has a point of action such as a leaf spring to be supported.

<1-2-2. Configuration of drive unit for high-speed drive>
On the other hand, as will be described later, since the image is taken again after shifting pixels, it is preferable that the shooting interval be as short as possible. This is because if there is a difference in the captured image due to subject blurring or camera shake, it will not only be difficult to recover by image processing, but it may also cause degradation in image quality, so it is necessary to minimize the effect. Because.

  Therefore, in order to increase the pixel shift drive speed, in the image processing apparatus 1, the drive unit 23 is configured by a piezoelectric element that expands and contracts along the optical axis direction. A piezoelectric element has a very high resonance frequency of about several tens of kilohertz to several megahertz, so that it can be driven up to the resonance frequency and has a large amount of displacement, so that a displacement necessary for tilting is generated at high speed. Because it can.

<1-2-3. Arrangement of tilting device for thinning>
In order to reduce the thickness, the tilting device is arranged in the direction orthogonal to the optical axis direction instead of arranging the driving unit 23 on the bottom surface or the top surface of the main body CM1 of the imaging module CM as described above. It is preferable to arrange the portion 23. This is because if the method of arranging the piezoelectric elements on the bottom surface or the top surface of the main body CM1 of the imaging module CM is employed, the thickness is increased in the optical axis direction.

  Therefore, in order to avoid the thickness in the optical axis direction, in the image processing apparatus 1, the formation length D (see FIG. 2B) of the drive unit 23 in the optical axis direction is the formation in the optical axis direction of the imaging module CM. It is configured to be shorter than the length T. Further, as shown in FIG. 2C, the fulcrum portion 24 is attached by a leaf spring or the like (not shown) in the direction of the arrow AR3 in order to prevent the fulcrum portion 24 from moving when driven by the drive portion 23. However, the height D ′ of the fulcrum 24 in the optical axis direction (see FIG. 2B) is also configured to be shorter than the formation length T of the imaging module CM in the optical axis direction.

<1-2-4. Modification of tilting device>
As a modification of the tilting device 22, the drive unit 23 may be configured as the drive unit 23 ′ of the tilting device 22 ′ shown in FIG. That is, as shown in FIG. 3A, in the image capturing device 1 ′, the drive portion 23 ′ is in contact with a lower portion of the extension portion CM2 and extends in the + X direction, and the extension portion 23b is formed. And a fixing portion 23c for fixing the + X side tip of the installation portion 23b. Here, the extending portion 23b of the driving portion 23 ′ is constituted by a kind of piezoelectric element such as a bimorph, a unimorph, and a piezoelectric unimorph, and expands and contracts in the direction of the arrow AR4 as shown in FIG. 3B. Regarding the expansion and contraction of the extending portion 23b in the direction of the arrow AR4, as shown in FIG. 3C, by applying a voltage from zero to the periphery of the −X side tip of the extending portion 23b, the extending portion 23b The −X side tip periphery of the wire portion contracts and deforms by warping in the + Z direction as indicated by the broken line position L1 from the solid line position L0, and the −X side tip periphery of the extending portion 23b is changed from the state of the broken line position L1. Is lowered, the vicinity of the −X side tip of the extending portion 23b is extended, and as shown in FIG. 3D, it is warped and deformed in the −Z direction as shown by a solid line position L2. Note that the formation length D ″ of the driving unit 23 ′ in the optical axis direction (here, corresponding to the length of the fixing unit 23c in the optical axis direction) is similar to the driving unit 23 in the optical axis direction of the imaging module CM. Is formed shorter than the formation length T.

<1-2-5. Effect of tilting device>
As described above, the image capturing apparatus 1 includes the imaging module CM including the imaging lens, the pixel-type imaging element SR that forms the subject image formed through the imaging lens during the imaging process, and the main body of the imaging module CM. It is necessary to change the imaging module CM by including a tilting device 22 (22 ′) that is disposed outside the CM1 and tilts the imaging module CM by a small amount with respect to a plane perpendicular to the optical axis to execute a tilting process. Therefore, it is possible to avoid an increase in the size of the imaging module CM, so that the overall configuration of the image capturing apparatus is simplified compared to a conventional apparatus that performs pixel shifting. Further, since the imaging module itself may be the same as the conventional one, the image quality of the original image capturing apparatus is not deteriorated.

  Further, the tilting device 22 (22 ′) directly transmits a displacement amount along the optical axis direction to a predetermined driven portion (extension portion CM2) of the imaging module CM, thereby executing a tilting process 23 (23). Therefore, the driving amount depends only on the displacement amount obtained from the physical constant and shape of the driving element. For this reason, it is not necessary to consider the drive error, the drive accuracy is improved, and high-precision repeatability can be realized. As a result, high-quality image acquisition is possible.

  Further, the drive unit 23 (23 ′) is composed of a piezoelectric element that expands and contracts along the optical axis direction, so that the processing speed of the tilting process can be increased and the photographing time can be shortened. Is possible. As a result, it is possible to reduce the influence of subject blur and camera shake.

  Further, the drive unit 23 (23 ') is arranged in a direction perpendicular to the optical axis with respect to the imaging module CM, so that the tilting process can be performed without increasing the thickness in the optical axis direction of the imaging module CM.

  Furthermore, since the formation length D (D ″) of the drive unit 23 (23 ′) in the optical axis direction is shorter than the formation length T of the imaging module in the optical axis direction, the drive unit 23 (23 ′) is newly added. Even if it is provided, it is not necessary for the entire apparatus to have a thickness in the optical axis direction of the imaging module CM, and a thin configuration can be realized.

  As a result, it is possible to provide the image capturing device 1 that is thin and has a compact configuration and high image quality output.

<1-3. Diagonal pixel shift>
Next, oblique pixel shifting, which is a premise of the first embodiment, will be described. Hereinafter, in this specification, the half pixel means a half of the arrangement pitch which is a distance between adjacent pixels in a predetermined direction of the pixels of the image sensor SR. Therefore, the image photographing apparatus 1 employs the oblique pixel shifting method, and the pixels of the image sensor SR are composed of a plurality of pixels arranged in a regular square as shown in FIG. Is the diagonal direction of the square regular array, that is, the arrow ARS direction (hereinafter simply referred to as “oblique direction”).

  In the case of the pixel arrangement of the square regular arrangement, the pixel shift is a method of shifting a half pixel in two orthogonal directions (hereinafter referred to as “XY pixel shift”, details will be described later), and a half pixel shift in an oblique direction of the array pitch. There are various variations such as a method (hereinafter referred to as “oblique pixel shift”, details will be described later), and a method of shifting in two orthogonal directions in the case of a hexagonal regular pixel configuration. In addition, the resolution can be further increased by shifting 1/3 pixel or 1/4 pixel instead of shifting by half a pixel. Since the amount of information that can be recovered increases as the number of shootings increases, the resolution can be increased. It is known that image quality (frequency information) is improved.

  However, when shooting with a normal portable camera is assumed, the effects of subject blurring and camera shake cannot be avoided. Therefore, it is difficult to say that increasing the number of shootings is generally suitable for higher image quality.

  In the image capturing device 1 according to the present embodiment, the drive unit 23 (23 ′) is configured so that the pixel position of the image sensor SR with respect to the subject image is the distance between adjacent pixels in an oblique direction before and after the tilt process. The position is changed in an oblique direction (that is, half a pixel in the oblique direction).

  This makes it possible to sample the subject image at a higher spatial frequency than determined by the pixel pitch. In addition, frequency information in two orthogonal directions in the square regular array can be doubled by performing the tilt process for changing the position in the diagonal direction of the square regular array only once. For this reason, the image quality can be improved uniformly regardless of the directionality of the subject. Furthermore, since it is only necessary to shoot twice with one half-pixel shift, the influence of subject blurring and camera shake is minimized, and the amount of movement is small compared to the amount of movement of an odd number of half-pixels. Therefore, it is possible to minimize the movement error.

  Specifically, image processing for shifting diagonal pixels realized by the image capturing apparatus 1 will be described. 5A and 5B are schematic diagrams for explaining image processing with diagonal pixel shifting. FIG. 5A shows the positional relationship between the first image and the second image, and FIG. 5B shows the first image and the second image. The positions of the pixels to be interpolated by analogy with the pixels of the second image are shown, and FIG. 5C shows the display image after the interpolation.

  First, as shown in FIG. 4B, when the position of the image sensor SR is moved by half a pixel in the arrow ARS direction by the diagonal pixel shifting process, as shown in FIG. From the pixel (hereinafter referred to as “first pixel”) P1 of the first image G1 that is the position of the pixel of the second image G2 (hereinafter referred to as “second pixel”) that is the position of the pixel after the tilting process. Moved to P2. Thereby, the information of the second pixel P2 is added to the information of the first pixel P1.

  Then, as shown in FIG. 5B, the pixel information of the intermediate position C0 is analogized from the information of the first pixel P1 and the information of the second pixel P2, as shown in FIG. 5C. By newly interpolating the analogy pixel P3, the display image GD is finally generated.

  As a result, the information of the analogy pixel P3 is further added to the first pixel P1 information and the second pixel P2 information, and as a result, the pixel information is four times that of the case of only the first pixel P1 information. Thus, a high resolution image is obtained.

<1-3-1. General properties and preconditions of pixel shifting by tilting>
However, the pixel shift processing by tilting has a problem that the pixel shift amount varies depending on the image height. The projection method of the normal imaging optical system has a relationship of y = f · tan θ, where y is the image height, f is the focal length, and θ is the angle of view. Because of the pixel shift, the amount of movement of the image height when tilted by a minute amount is described by the differentiation of θ, so dy / dθ = f / cos ² θ. That is, the amount of movement of the image height varies depending on the incident angle (image height).

  Table 1, FIG. 6 and FIG. 7 show the center of the image sensor SR in the optical system (focal length f = 3.5 mm, screen diagonal 4.04 mm) with a half field angle of 30 degrees (θ = 60 degrees). It shows the variation of the pixel shift amount due to the image height (y) when 0.5 pixel shift is performed.

  Table 1 numerically shows the relationship between the angle of view (θ), the image height (y: distance from the center of the image sensor SR), dy / dθ, and the amount of pixel shift. FIG. 5 is a graph showing the relationship between the pixel shift amount and the image height based on the graph. FIG. 7 is a conceptual diagram for explaining that the amount of pixel shift differs depending on the position of the image sensor SR.

  As shown in Table 1 and FIGS. 6 and 7, the movement of 0.5 pixels at the center of the image sensor SR is 0.67 pixels at the outermost periphery. That is, in the case of tilting, a phenomenon occurs in which the amount of pixel shift varies depending on the position of the image sensor SR. As shown in FIG. 7, the pixel shift amount increases in a rotationally symmetric manner toward the outside from the center of the image sensor SR, and the increase rate becomes steeper as it goes to the periphery. For this reason, if the same image processing is performed in all regions, the image quality is deteriorated.

  For this reason, in order to balance the entire image, a method of distributing the image sensor SR so that the center of the image sensor SR is 0.45 pixels or the like so that the entire image is close to 0.5 pixels (hereinafter referred to as “distribution method”) can be used. However, depending on the movement amount of the pixel shift, there are cases where this method cannot be used.

<1-3-2. High resolution processing>
Therefore, in the image capturing apparatus 1 of the present embodiment, pixel shift due to tilting has different pixel shift amounts depending on the position, so that high-resolution image processing is not uniform as a whole, and different processing is performed for each screen position. Preferably it is done.

  Hereinafter, the high resolution processing realized by the image capturing apparatus 1 will be described. FIG. 8 is a schematic diagram for explaining that different processing is performed for each position of the image sensor SR. FIG. 8A is an image in the vicinity of the center of the image sensor SR (when the pixel is shifted by 0.5 pixels in an oblique direction). FIG. 8B is an example of processing, and FIG. 8B is an example of image processing in the vicinity of the outermost periphery of the image sensor SR (when the pixel is shifted by 0.7 pixels in an oblique direction).

  In the vicinity of the center of the image sensor SR, as shown in FIG. 8A, when the diagonal direction of the square regular array is 1, the pixels are evenly shifted by 0.5 pixels. For this reason, in the case of pixel interpolation, as described above, the distance between the first pixel P1 of the first image G1 and the second pixel P2 of the second image G2 is always equal. The analog pixel at the center position C0 can be calculated from the information of the two pixels P2 by a uniform process (for example, a simple average of the upper and lower first pixels P1 and the left and right second pixels P2), and the interpolation method is extremely Simple.

  However, in the vicinity of the outermost periphery of the image sensor SR, as shown in FIG. 8B, the image pickup positions are not uniform as shown in FIG. In that case, instead of simple averaging, interpolation by weighted average (hereinafter referred to as “weighted interpolation”) in which information on pixels close to the pixel to be interpolated is increased and information on distant pixels is decreased. preferable. That is, in the case of FIG. 8B, when estimating the center position C1, the pixel information of the first pixel P11 and the second pixel P21 is increased, and the pixel information of the first pixel P12 and the second pixel P22 is increased. Interpolation to reduce Further, when correcting the position of the second pixel P21 to the center position C2, interpolation may be performed by adding more information about the first pixel P11 to the surrounding first pixels P12, P13, and P14.

  In the example of FIG. 8, the center of the image sensor SR is shifted by 0.5 pixels. However, weighting interpolation may be employed after adopting a sorting method to reduce the difference in the total pixel shift amount. Is possible.

  Therefore, in the image capturing device 1, the arithmetic processing unit 36 of the image processing unit 3 performs image processing in which the amount of pixel shift is considered according to the position of the image sensor SR (that is, the positions of the first and second images G1 and G2). To implement. That is, by performing weighted interpolation according to the positions of the first and second images G1 and G2, high-resolution processing that performs optimal processing in all regions is performed. Specifically, it is preferable to divide the screen into regions according to the same amount of pixel shift and use different interpolation coefficients for each region.

<1-4. Drive speed>
As described above, the drive speed in the drive unit 23 (23 ′) is preferably close to zero because of the influence of camera shake and the like. However, if the speed is increased more than necessary, the influence of damping vibration after driving may be This is not preferable because the tilting device becomes large. When there is a hand shake or the like corresponding to the driving amount during driving of the driving unit 23 (23 ′), the meaning of driving is lost. Therefore, the drive speed in the drive unit 23 (23 ′) will be considered below.

  FIG. 9 shows a schematic diagram of the frequency distribution of the 1 / 32-second camera shake angle acquired with a little less than 100,000 samples in handheld shooting of a compact camera. In addition, since the camera shake angle shows substantially the same distribution in both the pitch direction and the yaw direction, it is also shown in the distribution of FIG. As shown in FIG. 9, it can be seen that a peak is reached at a camera shake angle of about 0.025 degrees, and then abruptly decreases. In addition, since the angle is within 0.1 degrees, the camera shake angle at 30 ms can be regarded as a maximum of about 6 minutes. In an example described later (see Table 2 below), the pixel shift tilt angle assumed in the present embodiment is shown. As indicated by Condition 2-8 in Table 2 below, the tilt angle of pixel shift is in the range of about 0.5-1 minutes, so when the tilt angle is about 1 minute, it corresponds to about 5 ms. become. For this reason, when the influence of camera shake is taken into consideration, the drive speed of the drive unit 23 (23 ') needs to be 5 ms or less. On the other hand, the driving error corresponds to 2.5 ms when the tilt angle is about 0.5 minutes, and therefore it is considered that a tolerance of about 10% can be allowed. Side effects such as an increase in size are problematic.

  Therefore, in the image capturing device 1, the time required for one tilting process by the drive unit 23 (23 ') of the tilting device 22 (22') is set to 0.25 ms or more and 5 ms or less. That is, the influence of camera shake can be minimized by setting the time required for one tilting process by the drive unit to be 5 ms or less. In addition, by setting the pixel shift drive time of the drive unit to 0.25 ms or more, it is possible to avoid the influence of damping vibration after driving and the enlargement of the tilting device.

<1-5. Basic Operation of Image Shooting Apparatus 1>
FIG. 10 is a flowchart for explaining a basic operation realized in the image photographing apparatus 1 according to the present embodiment. Since the individual functions of each unit have already been described (see FIG. 1), only the overall flow will be described here. In FIG. 10, the same process is performed regardless of which of the tilting devices 22 and 22 ′. Therefore, the following description will be given on the assumption that the tilting device 22 is used.

  As shown in FIGS. 1 and 10, first, in step S1, the control unit 20 gives an instruction to the imaging module CM to take the first image, and the imaging module CM displays the first image G1 of the subject (FIG. 5 ( Take a).

  In step S <b> 2 </ b> A, the control unit 20 instructs the drive unit 23 of the tilting device 22 to perform a tilt process, and the drive unit 23 performs the tilt process. Thereby, as shown in FIG. 4B, the position of the image sensor SR is moved by half a pixel in the direction of the arrow ARS.

  In step S2B executed in parallel with step S2A, the first image G1 captured by the imaging module CM in step S1 is output to the image processing unit 3 via the interface 37. Then, the image processing unit 3 stores the input first image G1 in the storage device 34.

  In step S3, the control unit 20 gives an instruction to the imaging module CM to perform the second imaging, and the imaging module CM captures the second image G2 (see FIG. 5A) of the subject.

  In step S4, the imaging module CM outputs the second image G2 captured in step S3 to the image processing unit 3. Then, the image processing unit 3 stores the input second image G2 in the storage device 34 via the interface 37.

  In step S <b> 5, the arithmetic control unit 36 performs image processing using the first and second images G <b> 1 and G <b> 2 stored in the storage device 34. Specifically, as the image processing method, as described above, the high resolution processing (see FIG. 8) is performed by performing weighted interpolation incorporating a weighted interpolation and a sorting method.

  In step S6, the arithmetic control unit 36 generates a display image GD (see FIG. 5C) finally obtained through the high-resolution processing in step S5.

  In step S <b> 7, the obtained display image GD is visually output to the display unit 28 according to a command from the control unit 20.

<2. Second Embodiment>
In the first embodiment, the diagonal pixel shift process by one movement is performed as the pixel shift process, but in the second embodiment, the XY pixel shift process by three movements is performed.

  The schematic configuration and function of the image capturing device 1A in the second embodiment are different from those of the first embodiment in that the tilting device 22 (that is, the drive units 23 and 23 ′) is tilted as shown in FIG. By changing to (that is, the driving unit 22A), the control unit 20A and the image processing unit 3A (that is, the calculation control unit 36A) are changed as functions corresponding to the three tilt processes by shifting the XY pixels. The remaining configuration is the same as that of the apparatus of the first embodiment.

  FIG. 11 is a diagram illustrating an appearance of the image capturing device 1A according to the second embodiment. FIG. 11A illustrates the image capturing device 1A with the opening 25 of the main body CM1 of the imaging module CM facing upward (+ Z direction). FIG. 11B is a perspective view of the image capturing apparatus 1A. As shown in FIG. 11, the specific configuration of the tilting device 22A is different from that of the first embodiment, but the remaining configuration is the same as that of the first embodiment. The tilting device 22A has a configuration similar to the tilting device 22 'in the first embodiment, but differs from the tilting device 22' in the first embodiment in the following points.

  First, (i) the tilting device 22 'has a single drive unit 23', whereas the tilting device 22A is different in that the drive units 23AX and 23AY are provided in two orthogonal directions. Further, (ii) in the tilting device 22 ′, the extending portion CM2 is positioned in the direction of the arrow AR1 (square diagonal direction) of the imaging module CM in relation to the fulcrum portion 24, whereas in the tilting device 22A, the extending portion CM2x. The fulcrum part 24AX is located at the center of the short sides (Y direction) facing each other in the main body part CM1 of the imaging module CM, and the straight line connecting the two is parallel to the X axis. The extension part CM2y and the fulcrum part 24AY are In the main body part CM1 of the imaging module CM, a straight line located at the center of the long sides (X direction) facing each other is parallel to the Y axis. Further, (iii) the lengths of the extending portions 23bx and 23by become shorter than the extending portion 23b because the positions of the extending portions CM2x and CM2y are different from the positions of the extending portions CM2.

  12 and 13 are schematic views for explaining image processing with XY pixel shift. Note that the half pixel in the image capturing apparatus 1A that employs the XY pixel shifting method means half of the arrangement pitch that is the distance between adjacent pixels in the X direction or the Y direction of the pixels of the image sensor SR. That is, as shown in FIG. 12, when the pixels of the image sensor SR are composed of a plurality of pixels arranged in a square order, the half pixel in the X direction is the arrow ARX1 (ARX2) (see FIG. 12A and FIG. 12). 12 (c)), and the half pixel in the Y direction is the arrow ARY direction (see FIG. 12B).

  As shown in FIG. 13A, the first image G1 is taken by the imaging module CM in accordance with a command from the control unit 20A (see FIG. 13A). Then, as shown in FIG. 11, a voltage is applied to the vicinity of the tip on the + Y side of the extending portion 23bx of the drive portion 23AX by the control portion 20A to fix the fulcrum portion 24AX (the fulcrum portion 24AY is released from the fixed state). In this state, the extending portion 23bx contracts, so that the extending portion CM2x tilts the imaging module CM. As a result, as shown in FIG. 12A, the image sensor SR moves by half a pixel in the arrow ARX1 direction (+ X direction), and the imaging module CM receives the first image G2A (FIG. 13 ( Shoot b).

  Subsequently, the control unit 20A applies a voltage near the −X side tip of the extension part 23by of the drive part 23AY to fix the fulcrum part 24AY (the fulcrum part 24AX is released from the fixed state). The extension part CM2y tilts the imaging module CM by contracting the vicinity of the tip on the −X side of 23by (see FIG. 11). As a result, as shown in FIG. 12B, the imaging element SR moves by half a pixel in the arrow ARY direction (−Y direction), and the imaging module CM receives the first image G2B (FIG. 13) according to a command from the control unit 20A. (C) is taken.

  Further, a voltage is applied to the vicinity of the + Y side end of the extending portion 23bx of the driving portion 23AX by the control unit 20A, and the vicinity of the + Y side end of the extending portion 23bx contracts while the fulcrum portion 24AX is fixed. The extending part CM2x tilts the imaging module CM (see FIG. 11). As a result, as shown in FIG. 12C, the imaging element SR moves by half a pixel in the direction of the arrow ARX2 (−X direction), and the imaging module CM receives the first image G2C (FIG. 13) in response to a command from the control unit 20A. (D) is taken.

  Finally, the arithmetic control unit 36A performs image processing using the first image G1 and the second images G2A to G2C, and then generates a display image GD (see FIG. 13D).

<2-1. Basic Operation of Image Shooting Apparatus 1A>
14 and 15 are diagrams illustrating an operation flow of the image capturing apparatus 1A according to the second embodiment. In the second embodiment employing this XY pixel shift, unlike the one tilt process by the diagonal pixel shift of the first embodiment, a total of three tilt processes are performed, so that the following process is changed. Is done.

  First, in step SS1, the control unit 20A gives an instruction to the imaging module CM to perform the first imaging, and the imaging module CM captures the first image G1 (see FIG. 13A) of the subject.

  In step SS2A, the control unit 20A gives a command to perform a tilt process to the drive unit 23AX of the tilting device 22A, and the drive unit 23AX performs the tilt process. As a result, as shown in FIG. 12A, the position of the image sensor SR is moved by half a pixel in the direction of the arrow ARX1.

  In step SS2B executed in parallel with step SS2A, the imaging module CM outputs the first image G1 captured in step SS1 to the image processing unit 3A via the interface 37. Then, the image processing unit 3A stores the input first image G1 in the storage device 34.

  In step SS3, the control unit 20A gives a command to the imaging module CM to perform the second imaging, and the imaging module CM captures the second image G2A (see FIG. 13B) of the subject.

  In step SS4A, the control unit 20A instructs the drive unit 23AY of the tilting device 22A to perform a tilt process, and the drive unit 23AY performs the tilt process. Thereby, as shown in FIG. 12B, the position of the image sensor SR in FIG. 12A is moved by half a pixel in the direction of the arrow ARY.

  In step SS4B executed in parallel with step SS4A, the imaging module CM outputs the first image G2A captured in step SS3 to the image processing unit 3A via the interface 37. Then, the image processing unit 3A stores the input first image G2A in the storage device 34.

  In step SS5, the control unit 20A gives an instruction to the imaging module CM to perform the third imaging, and the imaging module CM captures the second image G2B (see FIG. 13C) of the subject.

  In step SS6A, the control unit 20A gives a command to perform the tilting process to the drive unit 23AX of the tilting device 22A, and the drive unit 23AX performs the tilting process. As a result, as shown in FIG. 12C, the position of the image sensor SR in FIG. 12C is moved by half a pixel in the direction of the arrow ARX2.

  In step SS6B executed in parallel with step SS6A, the imaging module CM outputs the first image G2B captured in step SS5 to the image processing unit 3A via the interface 37. Then, the image processing unit 3A stores the input first image G2B in the storage device 34.

  In step SS7, the control unit 20A gives a command to the imaging module CM to perform the fourth imaging, and the imaging module CM captures the second image G2C (see FIG. 13D) of the subject.

  In step SS8, the imaging module CM outputs the first image G2C captured in step SS7 to the image processing unit 3A via the interface 37. Then, the image processing unit 3A stores the input first image G2C in the storage device 34.

  In step SS9, the arithmetic control unit 36A performs image processing using the first and second A to 2C images stored in the storage device 34. Specifically, as the image processing method, the high-resolution processing is performed by performing weighted interpolation incorporating a weighted interpolation or a distribution method.

  In step SS10, the arithmetic control unit 36A generates a display image GD finally obtained through the high resolution processing in step SS9.

  In step SS11, the obtained display image GD is visually output to the display unit 28 in accordance with a command from the control unit 20A.

<3. Third Embodiment>
In image shooting, an electronic zoom function by trimming an image is generally known, but the number of pixels is reduced by performing trimming, so that the resolution is reduced as compared with an optical zoom. On the other hand, since the pixel shift by tilting has a relationship of y = f · tan θ, the pixel shift amount varies depending on the image height, that is, the angle of view as described above.

  Therefore, in the image capturing device 1B according to the third embodiment, a desired area is trimmed with the electronic zoom function, and high resolution processing is performed by shifting pixels only in the area (shifting diagonal pixels or shifting XY pixels). To do. As a result, when the pixel is shifted by half a pixel, an output equivalent to optical double zoom is possible, and when the pixel is shifted by 1/3, an output equivalent to optical triple zoom is possible. Therefore, in the trimming area of the image, in addition to correcting the influence of the pixel shift amount by the above-described high resolution processing, it is equivalent to an image captured using an optical zoom unit that optically changes the angle of view. It is possible to obtain an image having a resolution of 5 as the display image GD.

  FIG. 16 illustrates a schematic functional configuration realized in order to acquire a high resolution image by combining the electronic zoom function in the image capturing device 1B according to the third embodiment of the present invention. As described above, the functional configuration is different from that of the first embodiment (or the second embodiment), and as described above, by adding the electronic zoom unit 38, the control unit 20B and the image processing unit 3B (that is, arithmetic control). The section 36B) is changed as a function corresponding to the electric zoom. The remaining configuration is the same as that of the apparatus of the first embodiment (or the second embodiment) (see FIG. 1).

  As shown in FIG. 16, the electronic zoom unit 38 cuts out a part of the image formed on the image sensor SR and changes the photographing magnification. Then, the arithmetic control unit 36B selectively executes image processing in the area cut out by the electronic zoom unit 38.

<3-1. Basic Operation of Image Shooting Apparatus 1B>
Subsequently, FIG. 17 is a diagram illustrating an operation flow of the image capturing device 1B according to the third embodiment. In the third embodiment, the electronic zoom unit 38 that did not exist in the first and second embodiments is added, and thus the following process is changed. In the third embodiment, any of the tilting devices 22, 22 ′ and 22A can be adopted. However, in FIG. 17, it is assumed that the first embodiment (tilting devices 22, 22 ′) is used. This will be described below.

  First, as shown in FIG. 17, in steps ST <b> 1 to ST <b> 4, processes similar to steps S <b> 1 to S <b> 4 of the first embodiment are performed.

  As described above, after the first and second images G1 and G2 are stored in the storage device 34, the first image G1 (or the second image G2) is once displayed on the display unit 28.

  Then, before proceeding to step ST5, the operator inputs the electronic zoom information to the image capturing device 1B via the operation unit 21. Then, the control unit 20B inputs the electronic zoom information to the electronic zoom unit 38.

  The “electronic zoom information” here is information for specifying an area that the operator wants to zoom in the image. If it is desired to always zoom a predetermined specific area of the image, it can be automatically set by inputting the information in advance.

  In step ST5, the electronic zoom unit 38 cuts out a part of each of the first and second images G1 and G2 based on the input electronic zoom information and changes the shooting magnification.

  Here, it is preferable that the electronic zoom unit 38 stores only the areas of the cut out first and second images G1 and G2 in the storage device 34. This is because the storage capacity can be reduced when only the trimming region is stored as compared with the case where the entire first and second images G1 and G2 are stored.

  In step ST6, the arithmetic control unit 36B performs image processing using the trimming areas of the first and second images G1, G2 stored in the storage device 34. Specifically, as an image processing method, high resolution processing is performed by performing weighted interpolation or the like in the trimming regions of the first and second images G1 and G2.

  In step ST7, the arithmetic control unit 36B generates a display image GD in the trimming region finally obtained through the high resolution processing in step ST6.

  In step ST8, the obtained display image GD is visually output to the display unit 28 in response to a command from the control unit 20B.

<4. Example>
Table 2 below shows an image photographing apparatus according to the present invention based on the image photographing apparatuses 1 and 1A shown in FIGS. 2 and 3, in which the configuration and functions of the image pickup lens, the image pickup element, and the drive unit are appropriately changed. Illustrate.

  As shown in Table 2, as the configuration of the imaging lens, (condition 1-1) focal length f (mm), (condition 1-2) F value indicating the brightness of the lens, (condition 1-3) imaging The formation length T (mm) in the optical axis direction in the main body CM1 of the imaging module CM related to the lens thickness, and (condition 1-4) the total angle of view θ (degrees) were changed.

  As the configuration of the image sensor, (Condition 2-1) Pixel pitch (μm), (Condition 2-2) Number of pixels in X (horizontal) direction, (Condition 2-3) Number of pixels in Y (vertical) direction (Condition 2-4) Diagonal (mm) was changed.

  Further, the configuration and function of the drive unit include (Condition 3-1) Element type, (Condition 3-2) Pixel shift method, (Condition 3-3) Length of element in optical axis direction (mm), (Condition 3-4) Applied voltage (V), (Condition 3-5) Distance between drive unit and fulcrum (mm), (Condition 3-6) Pixel shift amount (μm), (Condition 3-7) The displacement amount (mm) of the drive unit (element) in the optical axis direction and (Condition 3-8) The tilt angle (minute) of the main body CM1 of the imaging module CM with respect to the horizontal plane were changed.

  (Condition 3-1) As for the type of element, Examples 1, 2, and 4 are configured using piezoelectric elements as in the case of the image capturing apparatus 1. On the other hand, in Examples 3 and 5, the piezo film is arranged below the main body CM1 of the imaging module CM, that is, on the flexible substrate 26. In the present invention, the tilting device is the main body of the imaging module CM. Since it is premised to be arranged outside the CM1, it does not depart from the technical scope of the present invention. Therefore, the predetermined driven portion in the third and fifth embodiments corresponds to the lower surface of the main body portion CM1 of the imaging module CM.

<5. Modification>
As mentioned above, although embodiment of this invention has been described, this invention is not limited to the said embodiment, A various deformation | transformation is possible.

  * In this embodiment, the image capturing devices 1 (1 ′), 1A, and 1B are described separately for each embodiment, but these individual functions are combined with each other unless they contradict each other. May be.

  * In the present embodiment, the drive units 23 and 23 'are configured using piezoelectric elements, but may be organic piezoelectric elements. Examples of the polymer organic piezoelectric material include, but are not limited to, polyvinylidene fluoride, a polyvinylidene fluoride-based copolymer, and polyvinylidene cyanide as a main component of vinylidene fluoride.

  The drive units 23 and 23 'may be giant magnetostrictive elements. The giant magnetostrictive element is preferable because it exhibits a large elastic deformation and a large mechanical force in a magnetic field. Examples of the giant magnetostrictive material include, but are not limited to, a single crystal giant magnetostrictive element made of terbium, dysprosium, and iron.

1, 1 ', 1A, 1B Image capturing device 3, 3A, 3B Image processing unit 20, 20A, 20B Control unit 22, 22', 22A Tilt device 23, 23 ', 23AX, 23AY Drive unit 23b, 23bx, 23by Extension Installation part 23c, 23cx, 23cy Fixed part 24, 24AX, 24AY Support point part 28 Display part 36, 36A, 36B Calculation control part 38 Electronic zoom part CM Imaging module CM2, CM2x, CM2y Extending part G1 First image G2, G2A, G2B , G2C second image

Claims (9)

An imaging module having a main body unit including an imaging lens and a pixel-type imaging element that forms a subject image formed through the imaging lens during imaging processing;
A tilting device that is disposed outside the main body of the imaging module and tilts the imaging module with respect to a plane perpendicular to the optical axis to perform a tilting process;
The tilting device is
A drive unit that executes the tilting process by directly transmitting a displacement amount along the optical axis direction to a predetermined driven part of the imaging module;
Have
The drive unit is
Before and after the tilting process, the position of the image sensor with respect to the subject image is changed in the predetermined direction by an odd multiple of an array pitch that is a distance between adjacent pixels in the predetermined direction of the pixels. Features
Image shooting device. The image capturing device according to claim 1,
The drive unit is composed of a piezoelectric element that expands and contracts along the optical axis direction.
Image shooting device. The image capturing device according to claim 2,
The pixels of the image pickup device include a plurality of pixels arranged in a square pattern,
The predetermined direction includes a diagonal direction of the square regular array,
Image shooting device. The image photographing device according to claim 1 or 2,
The drive unit is arranged in a direction perpendicular to the optical axis with respect to the imaging module,
Image shooting device. The image capturing device according to claim 4,
The imaging module has an extending part formed to extend in a direction perpendicular to the optical axis from the side surface of the main body part,
The predetermined driven portion includes the extending portion,
The drive unit performs the tilting process by moving the extending unit in the optical axis direction by contacting from below the extending unit.
Image shooting device. The image capturing device according to claim 2,
The formation length in the optical axis direction of the drive unit is shorter than the formation length in the optical axis direction in the main body of the imaging module,
Image shooting device. The image photographing device according to claim 1 or 2,
The time required for one tilting process by the drive unit is 0.25 ms or more and 5 ms or less,
Image shooting device. The image capturing device according to claim 1,
Image processing means for executing image processing for processing an image formed on the imaging element during the imaging processing and outputting a display image to a predetermined display unit;
Control means for controlling the imaging processing for the imaging module, the tilt processing for the tilting device, and the image processing in the image processing means;
Further comprising
The image processing means includes
The first image formed on the image sensor by the photographing process before execution of the tilting process in the tilting device under the control of the control unit, and the imaging process by the imaging process after the tilting process is performed. Using the second image formed on the element, the image processing is executed to obtain the display image.
Image shooting device. The image capturing device according to claim 8,
Electronic zoom means for cutting out a part of an image formed on the image sensor and changing a photographing magnification;
Further comprising
The image processing means includes
Selectively executing the image processing in an area cut out by the electronic zoom means;
Image shooting device.

Images