JP6296914B2 - IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to an imaging apparatus that acquires a plurality of image data while changing a focal position of a focus lens, a control method for the imaging apparatus, and a program.

  Some imaging apparatuses such as digital cameras have a function of acquiring a plurality of image data and realizing an image quality that cannot be expressed by a single image. For example, the feature point of a plurality of image data captured by changing the focal position of the focus lens is extracted, and the weighted average is performed so as to increase the weight of the pixel with high sharpness of the corresponding pixel between the plurality of images, A technique for obtaining an image with a deep depth of field (depth combined image) by combining pixels is known (see Patent Document 1).

Japanese Patent No. 4678603

  According to the technique described in Patent Document 1, an image with a high depth of field can be synthesized. However, in the case of handheld shooting, particularly when the subject is at a short distance, the camera is affected by camera shake, Shooting is performed at a position deviated from a preset focal position on the side. For this reason, the focused part is not continuous, and the depth composite image may be an unnatural image partially blurred.

  The present invention has been made in view of such circumstances, and even when there is a camera shake, an imaging apparatus capable of capturing a focus position with respect to a subject within a stable change width, a control method for the imaging apparatus, And to provide a program.

In order to achieve the above object, an imaging apparatus according to a first invention includes an imaging unit that captures an image of a subject to acquire image data, a lens control unit that controls a focal position by moving a lens, the subject and the imaging and spacing variation detection unit for detecting the distance variations parts to the lens optical axis direction, and the imaging instruction section issuing instructions on SL lens control unit, the subject and the imaging portion of the lens detected by the distance change detecting unit The image magnification calculation unit for calculating the image magnification of the subject to be imaged from the variation in the interval in the optical axis direction and the focal position of the lens controlled by the lens control unit, and the image magnification of the subject in the plurality of image data, respectively. comprising an image size adjustment unit which adjusts the image size so that the same, and the imaging instruction section, the focal point of the lens as previously plurality sets the target focus for the subject, it becomes the target focal plane By instructing the position control, a plurality of image data is acquired, and the lens control unit is instructed to correct the deviation from the target focal plane with respect to the subject caused by the variation in the distance between the subject and the imaging unit. Instructs focus position control .

In the imaging device according to a second invention, in the first invention, the interval variation detection unit detects a variation in the interval by detecting a change in the position of the imaging device.
In the imaging apparatus according to a third aspect, in the first aspect, the interval variation detection unit estimates the interval variation by detecting a change in image magnification of the subject to be imaged.

According to a fourth aspect of the present invention, there is provided an image pickup apparatus according to the first aspect, further comprising an image composition unit that synthesizes image data with an increased depth of focus using a plurality of image data on the target focal planes.

An image pickup apparatus according to a fifth aspect of the present invention is the image pickup apparatus according to the first aspect, further comprising an image pickup moving unit that moves the image pickup unit in the optical axis direction so as to suppress a variation in the distance in the lens optical axis direction. The instructing unit changes the imaging instruction based on the movement information of the imaging moving unit in addition to the detection result of the interval variation detecting unit.

According to a sixth aspect of the present invention, there is provided a method for controlling an imaging apparatus, comprising: an imaging unit that captures an image of a subject to acquire image data; and a lens control unit that controls a focal position by moving a lens. , a distance variation detection step of detecting the distance variations to the lens optical axis direction of the subject and the imaging unit, and the imaging instruction step of issuing a instructions on SL lens control unit, the subject detected by the distance change detecting step And an image magnification calculating step for calculating an image magnification of a subject to be imaged from a variation in an interval of the imaging unit in the lens optical axis direction and a focal position of the lens controlled by the lens control unit, and the plurality of image data has an image size adjustment step of image magnification of an object to adjust the image size so that each becomes equal, and the photographing instruction step, the target focus for the subject A plurality of image data are set in advance, and the control of the focal position of the lens is instructed so as to reach each target focal plane. The lens control unit is instructed to control the focal position of the lens so as to correct the deviation from the focal plane .

A program according to a seventh aspect of the invention is a program for causing a computer of an imaging apparatus to include an imaging unit that captures an image of a subject to acquire image data, and a lens control unit that controls a focal position by moving a lens. a is a distance change detecting step of detecting the distance variations to the lens optical axis direction of the subject and the imaging unit, and the imaging instruction step of issuing a instructions on SL lens control unit, detected by the distance change detecting step An image magnification calculating step of calculating an image magnification of the object to be imaged from the variation in the distance between the subject and the imaging unit in the lens optical axis direction, and the focal position of the lens controlled by the lens control unit; image magnification of the object in the image data has an image size adjustment step of adjusting the image size so that equal respectively, and the photographing instruction step, By setting a plurality of target focal points for the subject in advance and instructing control of the focal position of the lens so that each target focal plane is obtained, a plurality of image data is acquired, and the interval between the subject and the imaging unit is changed. Instruct the computer to instruct the lens control unit to control the focal position of the lens so as to correct the deviation of the generated subject from the target focal plane .

  According to the present invention, it is possible to provide an imaging apparatus, a control method for the imaging apparatus, and a program capable of capturing an image with the focal position of the subject within a stable change width even when there is a camera shake.

1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. In the camera which concerns on one Embodiment of this invention, it is a figure which shows the positional relationship of a camera and a to-be-photographed object. In the camera 10 which concerns on one Embodiment of this invention, it is a figure which shows the change of the image magnification when the positional relationship of a camera and a to-be-photographed object changes. 6 is a graph showing the relationship between the lens focal position, subject position, and contrast when a plurality of images are taken for depth synthesis (when there is no camera shake) in the camera according to an embodiment of the present invention. 6 is a graph showing the relationship between the lens focal position, subject position, and contrast when a plurality of images are taken for depth synthesis (when there is camera shake) in the camera of one embodiment of the present invention. 6 is a graph showing the relationship between the lens focal position, subject position, and contrast when a plurality of images are taken for depth synthesis (when there is camera shake) in the camera of one embodiment of the present invention. 4 is a flowchart illustrating a main operation of the camera according to the embodiment of the present invention. 4 is a flowchart illustrating a main operation of the camera according to the embodiment of the present invention. It is a flowchart which shows the operation | movement of imaging | photography of the camera which concerns on one Embodiment of this invention. It is a flowchart which shows the operation | movement of the image processing of the camera which concerns on one Embodiment of this invention. It is a flowchart which shows the modification of imaging | photography operation | movement of the camera which concerns on one Embodiment of this invention.

  Hereinafter, an example applied to a digital camera as an embodiment of the present invention will be described. This digital camera has an imaging unit, and converts the subject image into image data by the imaging unit, and displays the subject image on a display unit arranged on the main body based on the converted image data. The photographer determines the composition and the photo opportunity by observing the live view display. During the release operation, image data is recorded on the recording medium. The image data recorded on the recording medium can be reproduced and displayed on the display unit when the reproduction mode is selected.

  In addition, when the depth synthesis mode is set, the camera sequentially moves the focus position of the focus lens and acquires a plurality of images for depth synthesis. When acquiring multiple images for depth synthesis, when moving in the optical axis direction of the photographic lens due to the influence of camera shake etc., this movement is detected, so that the influence of the movement is canceled based on this detection result, Controls the focal position of the focus lens of the photographic lens.

  FIG. 1 is a block diagram mainly showing an electrical configuration of a camera according to an embodiment of the present invention. This camera includes a camera body 100 and an interchangeable lens 200 that can be attached to and detached from the camera body 100. In the present embodiment, the photographing lens is an interchangeable lens type, but the present invention is not limited to this, and a digital camera of a type in which the photographing lens is fixed to the camera body may of course be used.

  The interchangeable lens 200 includes a photographing lens 201, a diaphragm 203, a driver 205, a microcomputer 207, and a flash memory 209, and has an interface (hereinafter referred to as I / F) 199 with a camera body 100 described later.

  The photographing lens 201 includes a plurality of optical lenses (including a focus adjustment focus lens) for forming a subject image, and is a single focus lens or a zoom lens. A diaphragm 203 is disposed behind the optical axis of the photographing lens 201. The diaphragm 203 has a variable aperture diameter, and controls the amount of the subject luminous flux that has passed through the photographing lens 201. The photographing lens 201 can be moved in the optical axis direction by a driver 205, and the focus position is controlled by moving the focus lens in the photographing lens 201 based on a control signal from the microcomputer 207, so that the zoom lens In some cases, the focal length is also controlled. The driver 205 also controls the aperture diameter of the diaphragm 203.

  The microcomputer 207 connected to the driver 205 is connected to the I / F 199 and the flash memory 209. The microcomputer 207 operates according to a program stored in the flash memory 209, communicates with a microcomputer 121 in the camera body 100 described later, and controls the interchangeable lens 200 based on a control signal from the microcomputer 121. Do. The microcomputer 207, the driver 205, and the like function as a lens control unit that moves the focus lens in the photographing lens to control the focal position.

  The microcomputer 207 acquires the focus position of the focus lens from a focus position detection unit (not shown), and acquires the zoom position of the zoom lens from zoom position detection (not shown). The acquired focal position and zoom position are transmitted to the microcomputer 121 in the camera body 100.

  The flash memory 209 stores various information such as optical characteristics and adjustment values of the interchangeable lens 200 in addition to the above-described program. The microcomputer 207 transmits these various types of information to the microcomputer 121 in the camera body 100. The I / F 199 is an interface for performing communication between the microcomputer 207 in the interchangeable lens 200 and the microcomputer 121 in the camera body 100.

  A mechanical shutter 101 is disposed in the camera body 100 on the optical axis of the taking lens 201. The mechanical shutter 101 controls the passage time of the subject luminous flux and employs a known focal plane shutter or the like. An image sensor 103 is disposed behind the mechanical shutter 101 and at a position where a subject image is formed by the photographing lens 201.

  In the image sensor 103, photodiodes constituting each pixel are two-dimensionally arranged in a matrix, and each photodiode generates a photoelectric conversion current corresponding to the amount of received light, and this photoelectric conversion current is applied to each photodiode. Charges are accumulated by the connected capacitor. A Bayer array RGB filter is arranged in front of each pixel. The image sensor 103 has an electronic shutter. The electronic shutter controls the exposure time by controlling the time from charge accumulation to charge readout of the image sensor 103. Note that the image sensor 103 is not limited to the Bayer array, and may of course be a stacked form such as Foveon (registered trademark). The imaging element 103 functions as an imaging unit that captures an image of a subject and acquires image data. The imaging unit captures a subject a plurality of times and acquires a plurality of image data.

  The image sensor 103 is connected to an analog processing unit 105. The analog processing unit 105 performs waveform shaping on the photoelectric conversion signal (analog image signal) read from the image sensor 103 while reducing reset noise and the like. Further, the gain is increased so as to obtain a more appropriate luminance.

  The analog processing unit 105 is connected to an A / D conversion unit 107. The A / D conversion unit 107 performs analog-to-digital conversion on the analog image signal, and converts the digital image signal (hereinafter referred to as image data) to the bus 110. Output.

  The bus 110 is a transfer path for transferring various data read or generated in the camera body 100 to the camera body 100. In addition to the A / D conversion unit 107, the bus 110 includes an image processing unit 109, an AE (Auto Exposure) processing unit 111, an AF (Auto Focus) processing unit 113, an interval variation detection unit 115, an imaging instruction unit 117, A microcomputer 121, an SDRAM 127, a memory interface (hereinafter referred to as a memory I / F) 129, and a display driver 133 are connected.

  The image processing unit 109 includes a basic image processing unit 109a that performs normal image processing, an image composition unit 109b that performs image composition, and an image size adjustment unit 109c. When combining a plurality of images, the basic image processing unit 109a, the image combining unit 109b, and the image size adjusting unit 109c are used.

  The basic image processing unit 109a performs optical black (OB) subtraction processing, white balance (WB) correction, synchronization processing performed in the case of Bayer data, color reproduction processing, gamma correction processing, color matrix calculation, noise on RAW data. Reduction (NR) processing, edge enhancement processing, and the like are performed. When a single image is shot and no special effect such as art filter or depth composition is set, the image processing is completed only by the processing by the basic image processing unit 109a.

  The image composition unit 109b performs various image composition according to the set composition mode and the like. The image synthesizing unit 109b synthesizes image data using a plurality of image data acquired in different states such as a focus position and an aperture value. In the present embodiment, as will be described later, a composition mode such as depth composition for increasing the depth of the object field can be set. When the depth composition mode is set, the image composition unit 109b aligns the plurality of image data captured at the focal positions of the plurality of focus lenses, and the image has a high sharpness (contrast) area. Is extracted, and a region with a high degree of sharpness is synthesized to generate an image with a depth of field different from that of single shooting. The image synthesis unit 109b functions as an image synthesis unit that synthesizes image data with an increased depth of focus using a plurality of image data on each target focal plane.

  The image size adjustment unit 109c adjusts the image sizes of a plurality of images. The image size adjustment unit 109c adjusts the image size so that the image magnifications of a plurality of images are equalized before the image composition unit 109b performs depth composition to improve composition accuracy. The image size is adjusted by calculating the image magnification (see S123 and S127 in FIGS. 3 and 10), and multiplying the image by the calculated image magnification to make the image size the same. The image size adjustment unit 109c calculates the image magnification of the subject to be imaged from the interval variation in the lens optical axis direction between the subject and the imaging unit detected by the interval variation detection unit and the focal position of the lens controlled by the lens control unit. Functions as an image magnification calculator. The image size adjustment unit 109c functions as an image size adjustment unit that adjusts the image size so that the subject image magnifications in the plurality of image data are equal to each other (see S129 in FIG. 10).

  Although not shown, an image compression unit and an image expansion unit are provided in the image processing unit 109. The image compression unit compresses the image data read from the SDRAM 127 when recording the image data on the recording medium 131 according to various compression methods such as a JPEG compression method for a still image and MPEG for a moving image. . The image decompression unit also decompresses JPEG image data and MPEG image data for image reproduction display. In decompression, the file recorded in the recording medium 131 is read out, subjected to decompression processing in the image decompression unit, and the decompressed image data is temporarily stored in the SDRAM 127. In the present embodiment, a JPEG compression method or an MPEG compression method is adopted as an image compression method, but the compression method is not limited to this, and TIFF, H.264, or the like. Of course, other compression methods such as H.264 may be used. The compression method may be lossless compression or lossy compression.

  The AE processing unit 111 measures subject brightness based on image data input via the bus 110, and outputs the subject brightness information to the microcomputer 121 via the bus 110. Although a dedicated photometric sensor may be provided for measuring the subject brightness, in this embodiment, the subject brightness is calculated based on the image data.

  The AF processing unit 113 extracts a high-frequency component signal from the image data, acquires a focus evaluation value by integration processing, and outputs the focus evaluation value to the microcomputer 121 via the bus 110. In the present embodiment, the photographing lens 201 is focused by a so-called contrast method. In this embodiment, the AF control based on the contrast method has been described as an example. However, the subject luminous flux is divided and a phase difference sensor is provided on the optical path, or a phase difference sensor is provided on the image sensor, and AF control based on the phase difference AF is performed. You may also focus.

  The interval variation detector 115 detects an interval variation in the lens optical axis direction between the subject and the taking lens 201. Specifically, the interval variation detection unit 115 includes an acceleration sensor and the like, and detects the amount of movement of the camera body in the optical axis direction of the photographing lens 201. Note that the interval variation may be estimated from the variation in image magnification detected by the image size adjustment unit 109c. In this case, the image size adjustment unit 109c also functions as the interval variation detection unit 115.

  The interval variation detection unit 115 functions as an interval variation detection unit that detects an interval variation in the lens optical axis direction between the subject and the imaging unit. In addition, the interval variation detection unit detects the interval variation by detecting a change in the position of the imaging device (for example, position change detection by integrating acceleration change by the acceleration sensor, position change by GPS, etc.) (details) (See FIG. 2). The interval variation detection unit estimates the interval variation by detecting a change in the image magnification of the subject to be imaged (see FIG. 3 for details).

  The imaging instruction unit 117 receives the detection result from the interval variation detection unit 115, instructs the lens control unit to determine the focal position so as to suppress the interval variation due to camera shake or subject movement, and Instruct imaging. The shooting instruction unit 117 functions as a shooting instruction unit that issues a shooting instruction to the imaging unit and the lens control unit. The imaging instruction unit changes the imaging instruction according to the detection result of the interval variation detection unit. Further, the photographing instruction unit instructs the lens control unit to control the focal position of the lens so as to correct the deviation from the target focal plane with respect to the subject that occurs due to the change in the distance between the subject and the imaging unit (FIGS. 4 to 6). , See S109 in FIG. 9). In addition, the imaging instruction unit sets a plurality of target focal points for the subject in advance, and instructs the control of the focal position of the lens so that each target focal plane is obtained, thereby acquiring a plurality of image data (S101 in FIG. 9). , S115 etc.).

  The microcomputer 121 functions as a control unit for the entire camera, and comprehensively controls various sequences of the camera according to a program stored in the flash memory 125. In addition to the I / F 199 described above, an operation unit 123 and a flash memory 125 are connected to the microcomputer 121.

  The operation unit 123 includes operation members such as various input buttons and various input keys such as a power button, a release button, a moving image button, a playback button, a menu button, a cross key, and an OK button, and detects operation states of these operation members. The detection result is output to the microcomputer 121. The microcomputer 121 executes various sequences according to the user's operation based on the detection result of the operation member from the operation unit 123. The power button is an operation member for instructing to turn on / off the power of the digital camera. When the power button is pressed, the power of the digital camera is turned on. When the power button is pressed again, the power of the digital camera is turned off.

  The release button includes a first release switch that is turned on when the button is half-pressed and a second release switch that is turned on when the button is further depressed after being half-pressed and then fully pressed. When the first release switch is turned on, the microcomputer 121 executes a shooting preparation sequence such as an AE operation and an AF operation. When the second release switch is turned on, the mechanical shutter 101 and the like are controlled, image data based on the subject image is acquired from the image sensor 103 and the like, and a series of shooting sequences for recording the image data on the recording medium 131 are executed. Take a picture.

  The moving image button is an operation button for instructing start and end of moving image shooting. When the moving image button is first operated, moving image shooting is started, and when operated again, moving image shooting is ended. The playback button is an operation button for setting and canceling the playback mode. When the playback mode is set, the image data of the captured image is read from the recording medium 131 and the captured image is reproduced and displayed on the display panel 135.

  The menu button is an operation button for displaying a menu screen on the display panel 135. Various camera settings can be made on the menu screen. As camera settings, for example, there are synthesis modes such as depth synthesis, and the synthesis modes may include modes such as HDR synthesis and super-resolution synthesis.

  The flash memory 125 stores a program for executing various sequences of the microcomputer 121. The microcomputer 121 controls the entire camera based on this program.

  The SDRAM 127 is an electrically rewritable volatile memory for temporary storage of image data and the like. The SDRAM 127 temporarily stores image data output from the A / D conversion unit 107 and image data processed by the image processing unit 109 or the like.

  The memory I / F 129 is connected to the recording medium 131, and controls writing and reading of image data and data such as a header attached to the image data on the recording medium 131. The recording medium 131 is, for example, a recording medium such as a memory card that is detachably attached to the camera body 100, but is not limited thereto, and may be a hard disk or the like built in the camera body 100. The recording medium 131 functions as an image recording unit that records the composite image data.

  The display driver 133 is connected to the display panel 135, and displays an image on the display panel 135 based on the image data read from the SDRAM 127 or the recording medium 131 and expanded by the image expansion unit in the image processing unit 109. . The display panel 135 is disposed on the back surface of the camera body 100 and performs image display. The display panel 135 is a display unit that is easily affected by external light because the display surface is disposed on the exterior of the camera body such as the back surface, but a large display panel can be set. In addition, as a display part, various display panels, such as a liquid crystal display panel (LCD, TFT) and organic EL, are employable.

  As the image display on the display panel 135, a rec view display that displays recorded image data for a short time immediately after shooting, a playback display of a still image or moving image file recorded on the recording medium 131, a live view display, or the like. Video display.

  Next, with reference to FIG. 2 and FIG. 3, the effect of the change in the distance between the camera and the subject when the camera according to the present embodiment captures a plurality of images for depth synthesis will be described.

  FIG. 2 shows the camera 10 shooting the subject 20. FIG. 2A shows a state without camera shake. In FIG. 2A, the camera 10 is arranged on the imaging plane U, and the subject 20 is located on the target focal plane V with respect to the subject. Since the target focal plane V is at the lens focal position F of the photographing lens 201 of the camera 10, when the camera 10 photographs the subject 20 in this state, a focused image can be photographed.

  Assume that camera shake occurs in the state of FIG. 2A, and the camera 10 moves in the direction of the optical axis O by the amount of camera shake d as shown in FIG. 2B. At this time, the camera 10 moves to the imaging surface U ′ (imaging surface U ′ shifted due to camera shake in the optical axis direction). If the lens focal position F of the camera 10 does not change, the target focal plane V 'is brought into focus and the subject 20 on the target focal plane V becomes a blurred image.

  Therefore, in the present embodiment, as shown in FIG. 2C, the target focal plane of the camera 10 is changed to the lens focal position F ′ corrected by the camera shake amount d. That is, the camera shake amount d in the direction of the optical axis O of the camera 10 is detected, and the target focal plane is corrected with the camera shake amount d. For this reason, even if there is camera shake in the direction of the optical axis O, an in-focus image can be obtained.

  FIG. 3 shows a change in image magnification when the camera 10 moves in the optical axis direction due to camera shake. In FIG. 3A, the camera 10 is disposed on the imaging plane U, and the subject 20 is on the target focal plane V. In this case, the distance from the camera 10 to the target focal plane V is x, and at this time, the camera 10 can image the subject with a size X shown in FIG.

  Assume that the camera 10 has moved in the optical axis direction by the amount of camera shake d from the state of being imaged as shown in FIG. Since this movement is a relative change, FIG. 3A shows that the focal plane V has moved to the focal plane V ′. In this case, the distance between the camera 10 and the focal plane V ′ is xd because the camera 10 and the subject 20 are close to each other, and as shown in FIG. An image whose size is enlarged to X ′ is captured. Note that the range of angles + θ and −θ around the optical axis O is a range in which the size of the subject to be imaged is the same.

Now, assuming that the size of the subject 20 in the focal plane V is X and the size in the focal plane V ′ is X ′, the size X ′ can be calculated by the following equation (1). In addition, * in the following formula (1) means multiplication.
X ′ = X * x / (x−d) (1)

By using X and X ′ as the change in the image magnification of the subject from the above equation (1), the shift amount d in the optical axis direction can be calculated from the following equation (2).
d = (X′−X) · x / X ′ (2)

  Depending on the lens characteristics of the taking lens 201, the image magnification of the lens may vary depending on the focal position. In this case, it is necessary to calculate the amount of deviation in consideration of fluctuations in the image magnification of the lens. Strictly speaking, the size of the image sensor 103 also has an influence. The explanation here is based on the basic concept for easy understanding.

  As described above, the example shown in FIG. 2 is a case where the shift amount d in the optical axis direction of the photographic lens 201 is known. In this case, the target focal plane of the photographic lens 201 may be corrected with the shift amount d. . In the example shown in FIG. 3, the shift amount d in the optical axis direction is calculated from the image magnification fluctuation based on the change in the image size. If the amount of deviation d is known, the position of the target focal plane can be corrected using this amount of deviation d.

  Next, shooting for obtaining a plurality of image data for depth synthesis in the camera according to the present embodiment will be described with reference to FIGS.

  FIGS. 4A to 4C show a state in which there is no camera shake in the optical axis direction (Z-axis direction) of the photographing lens while photographing is repeatedly performed to obtain a plurality of image data for depth synthesis. Indicates. 4A shows time on the horizontal axis, the focal position of the lens on the vertical axis, and FIG. 4B shows time on the horizontal axis and subject position on the vertical axis. In the example of FIGS. 4A and 4B, at the times t1, t2, t3, and t4, the focal position of the photographing lens changes linearly, is at each point of P11, P12, P13, and P14, and the subject position is also It changes linearly and is at subject distances L11, L12, L13, and L14. In this state, shooting is performed at each timing of times t1, t2, t3, and t4. That is, the photographing instruction unit 117 performs photographing while shifting the lens focal position to the target positions (P11, P12, P13, P14).

  FIG. 4C shows the subject position on the vertical axis and the contrast of the image on the horizontal axis. In the example shown in FIG. 4, since there is no disturbance factor such as camera shake, the focal plane position with respect to the subject is photographed as intended. As can be seen from FIG. 4C, the contrasts of the images taken at the subject distances L11, L12, L13, and L14 overlap each other, and the threshold Th increases at any subject position. On the other hand, portions where the contrast is more than necessary (threshold Th) are continuous. When depth synthesis is performed using such four images, an in-focus image can be generated from the front to the back.

  FIGS. 5A to 5D show a state in which there is a camera shake in the optical axis direction (Z-axis direction) of the photographing lens during repeated photographing to obtain a plurality of image data for depth synthesis. Indicates.

  FIG. 5A also shows time on the horizontal axis, the focal position of the lens on the vertical axis, and FIG. 5B also shows time on the horizontal axis and the subject position on the vertical axis. In the example of FIG. 5A, as in the case of FIG. 4A, the focal position of the photographing lens changes linearly at times t1, t2, t3, and t4, and each of P11, P12, P13, and P14. In the point. However, since the position of the camera moves in the optical axis direction, the subject position from the camera 10 does not change linearly at each time, as shown in FIG. 5B, and subject distances L21, L22, L23 , L24. The amount of camera shake in the optical axis direction at this time is shown in FIG. In the example shown in FIG. 5B, the subject position is deviated from the target position by the amount of camera shake at this time.

  In this state, when shooting is performed at each timing of times t1, t2, t3, and t4, the position of the focal plane with respect to the subject is shifted even if the focal position of the lens is controlled as intended. FIG. 5C shows the subject position on the vertical axis and the contrast of the image on the horizontal axis, as in FIG. As can be seen from FIG. 5C, the contrasts of the images taken at the subject distances L21, L22, L23, and L24 overlap each other, but in some areas (area A in the figure) The contrast is also lowered by the threshold Th. For this reason, if depth synthesis is performed using these four images, the subject distance in the area A is not in focus and the image quality of the depth synthesis deteriorates.

  Therefore, in the present embodiment, camera shake in the optical axis direction is detected by a sensor such as an acceleration sensor of the interval variation detection unit 115, and the lens focus position is controlled so as to cancel this camera shake. That is, camera shake information is reflected in the lens focal position control.

  6A also shows the horizontal axis as time, the vertical axis as the lens focus position, FIG. 6B also shows the horizontal axis as time, the vertical axis as the subject position, and FIG. 6D shows the amount of camera shake in the optical axis direction. Shows changes. In the example shown in FIG. 6, the lens focal position is corrected so as to coincide with the target focal plane in accordance with the amount of camera shake detected by the interval variation detection unit 115 (see FIG. 6D), and P21, P22, Let P23 and P24. By setting the corrected positions P21, P22, P23, and P24 as lens focal positions, the subject positions are P11, P12, P13, and P14 as shown in FIG. 6B, and the distance between the camera and the subject is , According to the target. As a result, as can be seen from FIG. 6C, the contrast is higher than the threshold Th at any subject position, and portions where the contrast is obtained more than necessary (threshold Th) are continuous at each subject position. When depth synthesis is performed using such four images, an in-focus image can be generated from the front to the back.

  Next, the main processing of the camera in the present embodiment will be described using the flowcharts shown in FIGS. Note that the flowcharts shown in FIGS. 7 and 8 and FIGS. 9 to 11 described later are executed by the microcomputer 121 by controlling each unit in accordance with a program stored in the flash memory 125.

  When the power button in the operation unit 123 is operated and the power is turned on, the main flow shown in FIG. 7 starts its operation. When the operation is started, first, initialization is executed (S1). As initialization, electrical initialization such as mechanical initialization and initialization of various flags is performed. As one of the various flags, a recording flag indicating whether or not a moving image is being recorded is reset to OFF (see steps S13, S15, S31, etc.).

  Once initialization has been carried out, it is next determined whether or not the playback button has been pressed (S3). Here, the operation state of the playback button in the operation unit 123 is detected and determined. If the result of this determination is that the playback button has been pressed, playback / editing is executed (S5). Here, image data is read from the recording medium 131 and a list of still images and moving images is displayed on the LCD 135. The user operates the cross key to select an image from the list, and confirms the image with the OK button. In addition, the selected image can be edited.

  When playback / editing in step S5 is executed, or if the result of determination in step S3 is that the playback button has not been pressed, it is determined whether or not camera settings are to be made (S7). When a menu button in the operation unit 123 is operated, camera settings are performed on the menu screen. Therefore, in this step, determination is made based on whether or not this camera setting has been performed.

  If the result of determination in step S7 is camera setting, camera setting is carried out (S9). As described above, various camera settings can be performed on the menu screen. As camera settings, for example, normal shooting, depth synthesis, and the like can be set as shooting modes. In addition, the macro mode can be set. Here, the macro mode is a photographing mode suitable for photographing a subject at a short distance.

  If camera setting is performed in step S9, or if the result of determination in step S7 is not camera setting, it is next determined whether or not the movie button has been pressed (S11). Here, the microcomputer 121 inputs the operation state of the moving image button from the operation unit 123 and determines.

  If the result of determination in step S11 is that a movie button has been pressed, the recording flag is reversed (S13). The recording flag is set to on (1) during moving image shooting, and is reset to off (0) when no moving image is being shot. In this step, the flag is inverted, that is, when ON (1) is set, it is inverted to OFF (0), and when OFF (0) is set, it is inverted to ON (1). Let

  If the recording flag is reversed in step S13, it is next determined whether or not a moving image is being recorded (S15). Here, the determination is made based on whether the recording flag inverted in step S13 is set to ON or OFF.

  If the result of determination in step S15 is that movie recording is in progress, a movie file is generated (S19). In step S61, which will be described later, a moving image is recorded. In this step, a moving image file for moving image recording is generated, and preparation is made so that moving image data can be recorded.

  On the other hand, if the result of determination is that a moving image is not being recorded, the moving image file is closed (S17). Since the moving image button is operated and the moving image shooting is completed, the moving image file is closed in this step. When the moving image file is closed, the number of frames is recorded in the header of the moving image file so that the moving image file can be reproduced, and the file writing is ended.

  If the moving image file is closed in step S17, or if the moving image file is generated in step S19, or if the result of determination in step S11 is that the moving image button has not been pressed, it is next determined whether or not moving image recording is in progress. Perform (S31). In this step, as in step S15, the determination is made based on whether the recording flag is on or off.

  If the result of determination in step S31 is that moving image recording is not in progress, it is determined whether or not the release button has been half-pressed, in other words, whether or not the first release switch has been turned on from off (S33). This determination is made based on the detection result obtained by detecting the state of the first release switch linked to the release button by the operation unit 123. As a result of the detection, when the first release switch is changed from OFF to ON, the determination result is Yes. On the other hand, when the ON state or the OFF state is maintained, the determination result is No.

  If the result of determination in step S33 is that the release button has been half-pressed and transitioned from OFF to first release, AE / AF operation is executed (S35). Here, the AE processing unit 111 detects subject brightness based on the image data acquired by the image sensor 103, and calculates a shutter speed, an aperture value, and the like for appropriate exposure based on the subject brightness.

  In step S35, an AF operation is performed. Here, the driver 205 moves the focus position of the photographing lens 201 via the microcomputer 207 in the interchangeable lens 200 so that the focus evaluation value acquired by the AF processing unit 113 becomes a peak value. Accordingly, when the release button is pressed halfway when moving image shooting is not being performed, the shooting lens 201 is focused at that time and moved to the in-focus position. Thereafter, the process proceeds to step S37.

  If the result of determination in step S33 is that the release button has not transitioned from off to first release, it is next determined whether or not the release button has been fully pressed and the second release switch has been turned on (S41). . In this step, the state of the second release switch linked to the release button is detected by the operation unit 123, and a determination is made based on the detection result.

  If the result of determination in step S41 is that the release button has been fully pressed and the second release switch has been turned on, shooting is carried out (S43). Here, the aperture 203 is controlled by the aperture value calculated in step S33, and the shutter speed of the mechanical shutter 101 is controlled by the calculated shutter speed. When the exposure time corresponding to the shutter speed elapses, an image signal is read from the image sensor 103, and RAW data processed by the analog processing unit 105 and the A / D conversion unit 107 is output to the bus 110.

  When the depth synthesis mode is set, N target focal planes are set, and the focus lens of the photographing lens 201 is moved toward the target point. At this time, the interval variation detection unit 115 (specifically, an acceleration sensor) detects the amount of deviation in the direction of the optical axis O, corrects the position of the target focal plane based on this amount of deviation, and this correction is performed. When the target focal plane is reached, shooting is performed and image data is acquired. When N pieces of image data are acquired, the photographing in step S43 is terminated. Detailed operation of this photographing will be described later with reference to FIG.

  In step S43, image processing is performed after shooting (S45). RAW data acquired by the image sensor 103 is read, and image processing is performed by the image processing unit 109. If the depth synthesis mode is set, in step S43, depth synthesis is performed using a plurality of acquired image data. Detailed operation of this image processing will be described later with reference to FIG.

  Once image processing is performed, next still image recording is performed (S47). Here, the image data of the still image subjected to the image processing is recorded on the recording medium 131. When recording a still image, recording is performed in a set format (the recording format can be set in the camera setting in step S9). If JPEG is set, the image processed data is JPEG compressed and recorded in the image compression unit. In the case of the TIFF format, it is converted into RGB data and recorded in the RGB format. In addition, when RAW recording is set, when combining with RAW data acquired by photographing, combined RAW data is also recorded. The recording destination of the image data may be the recording medium 131 in the camera body, or may be recorded in an external device via a communication unit (not shown).

  If the result of determination in step S41 is not the release button 2nd, or if the result of determination in step S31 is that moving image recording is in progress, then AE is performed (S51). If the determination in Step S41 is No, it means that no operation is performed on the release button. In this case, live view display is performed in Step S57 described later. If the determination in step S31 is Yes, moving image recording is in progress. In this step, the shutter speed and ISO sensitivity of the electronic shutter of the image sensor 103 for performing live view display or moving image shooting with appropriate exposure are calculated.

  Once AE is performed, next, photographing with an electronic shutter is performed (S53). Here, the subject image is converted into image data. That is, charge accumulation is performed for an exposure time determined by the electronic shutter of the image sensor 103, and image data is acquired by reading the accumulated charge when the exposure time has elapsed.

  Once photographing with the electronic shutter is performed, next, image processing is performed on the acquired image data (S55). In this step, the basic image processing unit 109a performs basic image processing such as WB correction, color matrix calculation, gamma conversion, edge enhancement, and noise reduction.

  Once the basic image processing is performed, next, live view display is performed (S57). In this step, live view display is performed on the display panel 135 using the image data that has undergone the basic image processing in step S55. That is, since image data is acquired and image processing is performed in step S53, the live view display is updated using the processed image. The photographer can determine the composition and shutter timing by observing the live view display.

  If live view display is performed in step S57, it is next determined whether or not moving image recording is in progress (S59). Here, it is determined whether or not the recording flag is ON. If the result of this determination is that video recording is in progress, video recording is performed (S61). Here, the image data read from the image sensor 103 is subjected to image processing on moving image data and recorded in a moving image file.

  If moving image recording is performed in step S61, or if the result of determination in step S59 is that moving image recording is not being performed, still image recording is performed in step S45, or AE / AF is performed in step S35, It is determined whether or not it is off (S37). In this step, it is determined whether or not the power button of the operation unit 123 has been pressed again. If the result of this determination is that power is not off, processing returns to step S3. On the other hand, if the result of determination is that the power is off, the main flow is terminated after the main flow is completed.

  As described above, in the main flow in the embodiment of the present invention, it is possible to set a shooting mode for combining a plurality of image data such as a depth combining mode (S9), and when the depth combining mode is set, In step S43, the amount of deviation in the direction of the optical axis O is detected, the position of the target focal plane is corrected based on the amount of deviation, and shooting is performed when the corrected target focal plane is reached, and the image is used for depth synthesis. Get the data.

  Next, a detailed operation of photographing in step S43 will be described using the flowchart shown in FIG. In the shooting flow, first, N target focal planes are set (S101). Here, the imaging instruction unit 117 sets a target focal plane suitable for performing depth synthesis. That is, based on the focal length of the interchangeable lens 200, the aperture value of the aperture 203, whether or not close-up shooting is performed, a plurality of target focal planes that are sufficient and sufficient for depth synthesis are set, and the order of movement is also set. Set. Based on the set focus position, the lens control unit (microcomputer 207, driver 205, etc.) controls the position of the focus lens, and imaging is performed by the imaging unit (imaging device 103, etc.).

  If N target focal planes are set in step S101, then n = 1 is set (S103). This n is a variable for counting the number of times of photographing, is used for determination in step S115 described later, and is counted up in step S117.

  If n is 1 in step S103, then the nth target focal plane is set (S105). Here, the nth target focal plane among the target focal planes set in step S101 is set.

  If the nth target focal plane is set in step S105, next, blur in the optical axis direction is detected from the acceleration sensor (S107). Here, based on the output of the acceleration sensor in the interval variation detector 115, the amount of deviation of the taking lens 201 in the optical axis direction is detected. Note that the amount of misalignment in the optical axis direction is not limited to the acceleration sensor, but may be another sensor such as a gyro, and as described with reference to FIG. 3, a change in image magnification is detected from the image data. Alternatively, it may be obtained by calculating the shift amount d using the equation (2).

  If the blurring in the optical axis direction is detected in step S107, the lens focal position is controlled so as to become the nth target focal plane while correcting the focal plane shift due to the blurring (S109). Here, drive control of the focus lens in the photographic lens 201 is performed while correcting the nth target focal plane set in step S105 with the amount of deviation detected in step S107. That is, the lens control unit is instructed to control the focal position of the lens so as to correct the deviation from the target focal plane with respect to the subject in which the distance between the subject and the imaging unit varies.

  If the lens focus position is controlled in step S109, then photographing is performed (S113). Here, when the lens focal position corresponding to the target focal plane corrected in step S <b> 109 is reached, shooting is performed with the exposure control value calculated in step S <b> 35, and image data is acquired from the image sensor 103. In photographing here, the exposure time is controlled by the mechanical shutter 101, but it may be controlled by the electronic shutter of the image sensor 103. Image data acquired by shooting is temporarily stored in the SDRAM 127 or the like. Further, blur information (such as a shift amount) at the time of shooting and a lens focal position are temporarily stored in the SDRAM 127 or the like in association with image data.

  Once shooting has been performed in step S113, it is next determined whether or not the variable n is equal to N (S115). Here, it is determined whether or not N images set in step S101 have been taken.

  If n = N is not the result of the determination in step S115, n = n + 1, that is, 1 is added to the variable n (S117). When the variable n is counted up in step S117, the process proceeds to step S105, and imaging is repeated until N images are captured.

  If n = N as a result of the determination in step S115, the shooting flow is terminated and the process returns to the original flow.

  Thus, in the shooting flow in the present embodiment, when the focus lens is driven toward a plurality of target focal planes, the target focal plane is detected based on the amount of deviation in the optical axis direction from the interval variation detection unit 115. The lens focal position is controlled while correcting the deviation (see S109). For this reason, when shooting multiple images for depth synthesis, even if the distance between the camera and the subject fluctuates due to camera shake or the like, the subject is shot at the target focal plane as originally planned. can do.

  Next, the detailed operation of the image processing in step S45 will be described using the flowchart shown in FIG. In the image processing flow shown in FIG. 10, the description will focus on the processing for depth synthesis. In the image processing flow, first, the number of composites is set to N (S121). Here, the setting is made based on the number of shots N stored in the SDRAM 127.

  Once the composite number is set in step S121, the first image magnification is calculated from the blur information and the lens focal position at the time of the first photographing (S123). Since the blur information and the lens focal position information are recorded together with the image data at the time of shooting in step S113, the first image magnification is calculated using these pieces of information. The image magnification is calculated because the images for depth synthesis have different target focal planes at the time of shooting, and therefore the image magnification of each image is different, and therefore the size of the subject is different. . If depth synthesis is performed while the subject size remains different, the image quality deteriorates. Therefore, the subject size is adjusted using the image magnification, and then the depth synthesis is performed. In addition, in calculating the image magnification, the information on the lens focal position is taken into account because there is a lens whose image magnification varies depending on the lens focal position.

  If the first image magnification is calculated in step S123, then the variable n = 2 is set (S125).

  If n = 2 in step S125, then the n-th image magnification is calculated from the blur information and the lens focal position at the n-th shooting (S127). Since the blur information and the lens focal position information are recorded together with the image data at the time of shooting in step S113, the n-th image magnification is calculated using these pieces of information.

  Once the image magnification is calculated in step S127, the n-th photographed image is then adjusted in size so that the n-th image magnification is equal to the first image magnification (S129). Here, the image size adjustment unit 109c uses the image magnification calculated in step S127 to align the image size with the size of the first image.

  Once the image size is adjusted in step S129, the n-1th composite image and the nth captured image are combined to generate the nth composite image (S131). Here, the image composition unit 109b aligns the n-1st composite image and the nth image, extracts a region with high sharpness (contrast) of the image, and composes a region with high sharpness. Thus, an image having a depth of field is generated.

  If a composite image is generated in step S131, it is next determined whether n = N (S133). Here, it is determined whether or not depth composition has been performed using all of the N images set in step S121.

  If n = N is not the result of the determination in step S133, n = n + 1, that is, 1 is added to the variable n (S135). When the variable n is counted up in step S135, the process proceeds to step S127, and the image composition is repeated until the composition of N sheets is completed.

  If n = N as a result of the determination in step S133, the image processing flow is terminated, and the original flow is restored.

  As described above, in the image processing flow in the present embodiment, the image magnification is calculated for each image (S123, S127), and the image size is adjusted using the image magnification so that the image sizes are uniform. (S129). In the depth synthesis, since the image sizes are the same, a high-quality depth synthesized image can be generated.

  As described above, in one embodiment of the present invention, the influence of camera shake in the optical axis direction is corrected at the time of photographing, and the driving of the photographing lens 201 is controlled to the lens focal position corresponding to the target focal plane ( S109 of FIG. 9). For this reason, even when there is a blur, the focus position with respect to the subject can be captured within a stable change width.

  Next, a modified example of the photographing operation in step S43 will be described with reference to FIG. In one embodiment of the present invention, the image sensor 103 does not have a drive unit in the optical axis direction. However, in the present modification, the image sensor 103 has an imaging moving unit that drives the image sensor 103 in the optical axis direction. Yes. For this reason, if the image sensor 103 is moved in the optical axis direction so as to cancel out the shift amount in the optical axis direction detected when capturing a plurality of images for depth synthesis, it is possible to capture an image on the target focal plane. However, even in this case, the amount of deviation may not be completely corrected. Therefore, in the present modification, the remaining shift amount is calculated, and the lens focal position is controlled so as to be the target focal plane.

  In the configuration of this modification, an imaging moving unit that drives the image sensor 103 in the optical axis direction is provided in the block diagram shown in FIG. 1, and the microcomputer 121 controls drive of the imaging moving unit. The imaging movement unit moves the imaging unit in the optical axis direction so as to suppress the interval variation in the lens optical axis direction. Further, the imaging instruction unit 117 changes the imaging instruction from the movement information of the imaging moving unit in addition to the detection result of the interval variation detection unit 115. Since the other configuration is the same as that of the block diagram of FIG.

  The shooting flow in the present modification is the same as the shooting flow in FIG. 9 except that step S109 is replaced with steps S108 to S112, and therefore this difference will be mainly described.

  If the camera enters the shooting flow and detects a shake in the optical axis direction in step S107, the shake correction device is then operated (S108). Here, the imaging movement unit moves the imaging element 103 in the direction opposite to the blur direction so as to cancel out the blur amount in the optical axis direction.

  When the blur correction device is activated, the remaining blur amount is calculated from the detected blur amount and the blur amount corrected by the blur correction device (S110). The shake correction apparatus mechanically moves the image sensor 103 in the optical axis direction of the lens, and at this time, there may be a case where the shake correction cannot be performed completely. The remaining blur amount at this time is calculated.

  If the remaining blur is calculated in step S110, the lens focal position is then controlled so as to become the nth target focal plane while correcting the focal plane shift due to the remaining blur (S112).

  If the lens focal position is controlled in step S112, next, photographing is performed (S113). Since the following operations are the same as those in the shooting flow shown in FIG.

  As described above, in this modification, when a blur in the optical axis direction is detected, the position of the image sensor 101 in the optical axis direction is controlled based on the blur amount (S108), and the target focal plane is determined based on the remaining blur amount. By controlling the lens focal position so as to become (S110, S112), residual blur is removed. For this reason, a high-quality depth composite image can be obtained.

  As described above, in one embodiment or a modification of the present invention, an imaging unit (for example, the imaging device 103) that captures an image of a subject and acquires image data, and a focal point is controlled by moving a lens. A lens control unit (for example, a driver 205, a microcomputer 207, etc.), an interval variation detection unit (for example, an interval variation detection unit 115) that detects an interval variation in the lens optical axis direction between the subject and the imaging unit, and an imaging unit The imaging instruction unit (for example, the imaging instruction unit 117) that issues an imaging instruction to the lens control unit, and the imaging instruction unit changes the imaging instruction according to the detection result of the interval variation detection unit (for example, FIG. 9 S109). For this reason, even when there is camera shake, it is possible to shoot with the focus position with respect to the subject within a stable change width. In other words, even if the distance between the subject and the imaging unit varies, the distance is detected and the focal position of the lens is changed, so that an in-focus image can be obtained even when there is a camera shake. it can.

  In one embodiment or modification of the present invention, the interval variation detection unit (for example, the interval variation unit 115 in FIG. 1) detects the interval variation by detecting a change in the position of the imaging device. . It is possible to detect a change in the position of the imaging device by using the sensor output in the optical axis direction among the outputs of a gyroscope or an acceleration sensor provided for camera shake correction.

  In one embodiment or modification of the present invention, the interval variation detection unit estimates the interval variation by detecting a change in the image magnification of the subject to be imaged (for example, in FIGS. 3 and 1). Interval variation unit 115). For this reason, since the interval variation is estimated using the image data, it is not necessary to provide a sensor such as a gyroscope or an acceleration sensor.

  In one embodiment or a modification of the present invention, the shooting instruction unit causes the lens control unit to adjust the focus of the lens so as to correct the deviation from the target focal plane with respect to the subject caused by a change in the distance between the subject and the imaging unit. The position control is instructed (for example, S109 in FIGS. 5 and 9). For this reason, it is possible to perform control so that the subject is photographed at the target focal plane while canceling the influence of inadvertent interval fluctuations between the imaging unit and the subject.

  Further, in one embodiment or a modification of the present invention, the shooting instruction unit sets a plurality of target focal points for the subject in advance, and instructs the control of the focal position of the lens so as to be each target focal plane, A plurality of image data are acquired (for example, S101, S113, S115 in FIG. 9). For this reason, a plurality of image data can be acquired while changing the target focal plane for depth expansion.

  In one embodiment or modification of the present invention, an image composition unit (for example, the image composition unit in FIG. 1) that synthesizes image data with an increased depth of focus using a plurality of image data on each target focal plane. 109b). For this reason, a depth composite image for depth expansion can be generated.

  Further, in one embodiment or modification of the present invention, an image is picked up from the subject detected by the interval variation detection unit and the interval variation in the lens optical axis direction of the imaging unit and the focal position of the lens controlled by the lens control unit. An image magnification calculation unit (for example, the image size adjustment unit 109c in FIG. 1) that calculates the image magnification of the subject to be imaged and an image size adjustment that adjusts the image size so that the subject image magnifications in the plurality of image data are equal to each other (See, for example, the image size adjustment unit 109c in FIG. 1 and S129 in FIG. 10). For this reason, since it is possible to correct fluctuations in the image magnification of the captured image, it is possible to generate a high-quality depth composite.

  In the modification of the present invention, the imaging moving unit moves the imaging unit in the optical axis direction so as to suppress the interval variation in the lens optical axis direction, and the imaging instruction unit is detected by the interval variation detection unit. In addition to the result, the shooting instruction is changed from the movement information of the imaging moving unit (see S110, S112, etc. in FIG. 11). Even when a camera shake correction device is provided separately, since the focus position control of the lens is performed in consideration of the correction information, the focus position control with high accuracy can be performed.

  In one embodiment or modification of the present invention, an interval variation detecting step (for example, refer to S107 in FIG. 9) for detecting an interval variation in the lens optical axis direction of the subject and the imaging unit, an imaging unit, There is a photographing instruction step (S113 in FIG. 9) for issuing a photographing instruction to the lens control unit, and the photographing instruction step changes the photographing instruction according to the detection result of the interval variation detecting step (S109).

  In the embodiment and the modification of the present invention, a case has been described in which camera shake occurs, and therefore a change in the distance between the camera and the subject has occurred. However, the present invention is not limited to this, and the subject moves. For this reason, the present invention can also be applied to a case where a gap between the camera and the subject occurs.

  In one embodiment or modification of the present invention, the interval variation detection unit 115, the imaging instruction unit 117, the image processing unit 109, the AE processing unit 111, the AF processing unit 113, and the like are separated from the microcomputer 121. Of course, all or a part of each part may be configured by software and executed by the microcomputer 121.

  In the embodiment and the modification of the present invention, the digital camera is used as the photographing device. However, the camera may be a digital single lens reflex camera, a mirrorless camera, or a compact digital camera, and may be a video. Cameras for moving images such as cameras and movie cameras may be used, and cameras built in mobile phones, smartphones, personal digital assistants (PDAs), personal computers (PCs), tablet computers, game machines, etc. It doesn't matter. In any case, the present invention can be applied to any device that acquires a plurality of image data by changing a focal position in accordance with an imaging start instruction.

  Of the techniques described in this specification, the control mainly described in the flowchart is often settable by a program and may be stored in a recording medium or a recording unit. The recording method for the recording medium and the recording unit may be recorded at the time of product shipment, may be a distributed recording medium, or may be downloaded via the Internet.

  In addition, regarding the operation flow in the claims, the specification, and the drawings, even if it is described using words expressing the order such as “first”, “next”, etc. It does not mean that it is essential to implement in this order.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 10 ... Camera, 20 ... Subject, 100 ... Camera body, 101 ... Mechanical shutter, 103 ... Image sensor, 105 ... Analog processing part, 107 ... A / D conversion part, DESCRIPTION OF SYMBOLS 109 ... Image processing part, 109a ... Basic image processing part, 109b ... Image composition part, Image size adjustment part, 110 ... Bus, 111 ... AE processing part, 113 ... AF process , 115... Interval variation detection unit, 117... Photographing instruction unit, 121... Microcomputer, 123... Operation unit, 125... Flash memory, 127. Memory I / F, 131 ... Recording medium, 133 ... Display driver, 135 ... Display panel, 199 ... I / F, 200 ... Interchangeable lens, 201 ... Shooting lens, 203 ... Ri, 205 ... driver, 207 ... micro computer, 209 ... flash memory

Claims (7)

An imaging unit that images a subject and acquires image data;
A lens controller that controls the focal position by moving the lens;
An interval variation detection unit for detecting an interval variation in the lens optical axis direction of the subject and the imaging unit;
A photographing instruction unit to issue instructions to the top Symbol lens control unit,
Image magnification for calculating the image magnification of the subject to be imaged from the distance variation in the lens optical axis direction of the subject and the imaging unit detected by the interval variation detection unit and the focal position of the lens controlled by the lens control unit A calculation unit;
An image size adjustment unit that adjusts the image size so that the image magnifications of the subjects in the plurality of image data are equal to each other;
Comprising
The shooting instruction unit sets a plurality of target focal points for the subject in advance, and instructs the control of the focal position of the lens so that each target focal plane is obtained. An image pickup apparatus that instructs the lens control unit to control a focal position of a lens so as to correct a deviation from a target focal plane with respect to a subject that occurs due to a variation in an image pickup unit interval .   The imaging apparatus according to claim 1, wherein the interval variation detection unit detects an interval variation by detecting a change in a position of the imaging apparatus.   The imaging apparatus according to claim 1, wherein the interval variation detection unit estimates the interval variation by detecting a change in image magnification of a subject to be imaged. The imaging apparatus according to claim 1 , further comprising an image composition unit that synthesizes image data with an increased depth of focus using a plurality of image data on each target focal plane. An imaging movement unit that moves the imaging unit in the optical axis direction so as to suppress the interval variation in the lens optical axis direction;
The imaging instruction unit changes the imaging instruction from the movement information of the imaging moving unit in addition to the detection result of the interval variation detection unit;
The imaging apparatus according to claim 1. In a control method of an imaging apparatus, which includes an imaging unit that captures an image of a subject and acquires image data, and a lens control unit that controls a focal position by moving a lens.
An interval variation detecting step for detecting an interval variation in the lens optical axis direction of the subject and the imaging unit;
A photographing instruction step of issuing a instructions on SL lens control unit,
The image magnification for calculating the image magnification of the object to be imaged from the distance variation in the lens optical axis direction of the object and the imaging unit detected in the interval variation detection step and the focal position of the lens controlled by the lens control unit. A calculation step;
An image size adjustment step of adjusting the image size so that the subject image magnifications in the plurality of image data are equal to each other;
Have
The shooting instruction step sets a plurality of target focal points for the subject in advance, and instructs the control of the focal position of the lens so that each target focal plane is obtained, thereby obtaining a plurality of image data, A method for controlling an imaging apparatus, comprising: instructing the lens control unit to control a focal position of a lens so as to correct a deviation from a target focal plane with respect to a subject that occurs due to a change in an interval between imaging units. A program for executing a computer of an imaging apparatus having an imaging unit that captures an image of a subject and acquires image data, and a lens control unit that controls a focal position by moving a lens,
An interval variation detecting step for detecting an interval variation in the lens optical axis direction of the subject and the imaging unit;
A photographing instruction step of issuing a instructions on SL lens control unit,
The image magnification for calculating the image magnification of the object to be imaged from the distance variation in the lens optical axis direction of the object and the imaging unit detected in the interval variation detection step and the focal position of the lens controlled by the lens control unit. A calculation step;
An image size adjustment step of adjusting the image size so that the subject image magnifications in the plurality of image data are equal to each other;
Have
The shooting instruction step sets a plurality of target focal points for the subject in advance, and instructs the control of the focal position of the lens so that each target focal plane is obtained, thereby obtaining a plurality of image data, Instructing the lens control unit to control the focal position of the lens so as to correct the deviation from the target focal plane with respect to the subject that occurs due to the interval variation of the imaging unit .
A program characterized by causing a computer to execute the above.

Images