JP2008079123A - Imaging apparatus and focus control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve precision in focusing of an imaging apparatus. <P>SOLUTION: As an imaging apparatus, a digital camera 10 comprises: a CCD imaging element 16; a lens system 12; a motor 30 for focusing the lens system 12; a resolution improved region selection unit 24 and an image resolution improvement unit 26 for improving a resolution of the picked-up image of an arbitrary object using images of a plurality of objects picked up by the CCD imaging element 16; and a focal point detection unit 28 which detects a focal position of the lens system 12 using the resolution improved image and causes the motor 30 to focus the lens system 12 based on the detected focal position. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that electronically records an image of a subject and a focus control program for such an imaging apparatus.

  Conventionally, when autofocusing is performed in the focusing technique, an image is captured, the edge sharpness of the captured subject image is calculated, the focal length of the image is changed according to the sharpness, and the focus is changed. We were matching. Further, when performing manual focus, a captured subject image is displayed by an LCD viewer provided in the camera, and the user visually confirms the edge sharpness of the subject image to perform manual focus.

For example, in Patent Document 1, in the confirmation of the subject in the monitor display of the digital camera, when the release button is not pressed, the entire image of the subject captured by the image sensor is displayed on the liquid crystal display device provided in the digital camera. An invention for displaying an image of the area near the center of the focus area set in advance by the user in the enlarged area display at a higher resolution when the image is displayed at the resolution of the liquid crystal device and the release button is half pressed. It is disclosed.
JP 2003-179798 A

  Conventionally, in manual focus, in order for a user to visually confirm the sharpness of an edge of an image used for focusing, for example, the image is enlarged using the nearest neighbor method, and the sharpness of the edge is easily visible to the user. The technique to take is taken. In autofocusing, as a method for calculating the edge sharpness of an image used for focusing, for example, a method of calculating the edge sharpness by performing processing using a Laplacian filter in the vicinity of the edge is employed.

  However, in these conventional methods, since the edge sharpness is confirmed by enlarging only one image, there is a problem that the reliability of the edge sharpness evaluation is low and focusing cannot be performed with high accuracy. For example, in the case of manual focus, when an image is enlarged and displayed with an LCD viewer for visual confirmation of edge sharpness, the resolution of the image becomes insufficient because it is an enlargement method of only one image. There was a problem that manual focus could not be performed accurately. Also, for example, in the case of autofocus, since the enlargement method uses only one image when calculating the edge sharpness, autofocus is performed due to lack of resolution near the edge and the presence of noise. There is a problem that the evaluation value of the edge sharpness cannot be satisfactorily calculated.

  In particular, in the case of strong enlargement display, jaggy appears greatly and there is enlargement blur, so there may be a small change in sharpness before and after the in-focus position. In addition, although the sharpness can be confirmed in the image enlargement by the nearest neighbor method, since the appearance differs greatly between the real object and the enlarged image with a lot of block distortion, it is unnatural and is focused by the optical viewfinder. The user feels uncomfortable.

  The present invention has been made in view of the above points, and an object thereof is to provide an imaging apparatus and a focus control program capable of improving the focusing accuracy.

One aspect of an imaging apparatus of the present invention is an imaging apparatus that electronically records an image of a subject.
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
A focus detection means for detecting a focus position of the imaging optical system, using an image whose resolution is increased by the image resolution enhancement means;
An adjustment unit that causes the focusing unit to adjust the focus of the imaging optical system based on the in-focus position detected by the focus detection unit;
It is characterized by comprising.

An aspect of the imaging apparatus of the present invention is an imaging apparatus that electronically records an image of a subject.
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
Image display means for displaying an image having a high resolution in the image high resolution means;
Adjusting means capable of manually adjusting the focusing means;
It is characterized by comprising.

An aspect of the focus control program of the present invention is a focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
Using this high-resolution image, a procedure for detecting the in-focus position of the imaging optical system having the action of a lens,
A procedure for adjusting the focus of the imaging optical system based on the detected focal position,
Is executed.

An aspect of the focus control program of the present invention is a focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
The procedure to display this high resolution image,
A procedure for accepting an operation for adjusting the focus of the imaging optical system having the action of a lens;
A procedure for adjusting the focus of the imaging optical system in accordance with the operation;
Is executed.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device and focus control program which can improve the precision of focusing can be provided.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1 is a block configuration diagram showing a configuration of a digital camera 10 as an imaging apparatus according to the first embodiment of the present invention. The digital camera 10 includes a lens system 12, a color filter array (CFA) 14, a CCD image pickup device 16, an amplification circuit 18, an A / D conversion circuit 20, a buffer 22, a high resolution area selection unit 24, and an image high resolution unit. 26, an in-focus detection unit 28, a motor 30, a control unit 32, an external I / F unit 34, an image recording unit 36, and an output unit 38. Further, the image resolution increasing unit 26 includes a use number control unit 26A, a reference image selecting unit 26B, an image displacement estimating unit 26C, a resolution increasing target image selecting unit 26D, and a resolution increasing processing unit 26E.

  That is, in this digital camera 10, an image photographed through the lens system 12, the CFA 14, and the CCD image pickup device 16 is amplified by the amplification circuit 18, input to the A / D conversion circuit 20, and converted into a digital signal. Converted. The output from the A / D conversion circuit 20 is input to the buffer 22 via the video signal line. In the present embodiment, a single-plate CCD is assumed in which a Bayer-type primary color filter is arranged on the front surface in the imaging system. The Bayer-type primary color filter has 2 × 2 pixels as a basic unit, and a red (R) and blue (B) filter is arranged for each pixel, and a green (Gr, Gb) filter is arranged for two pixels. These green filters are the same, but in the present embodiment, these are distinguished from Gr and Gb for convenience.

  The output from the buffer 22 is input to the image recording unit 36 and the high resolution area selection unit 24 via the video signal line. The image recording unit 36 and the high resolution area selection unit 24 input to the image high resolution unit 26 via a video signal line, and the image recording unit 36 further outputs from the image high resolution unit 26. Can be entered. Outputs from the image resolution increasing unit 26 and the image recording unit 36 are input to the output unit 38 via a video signal line.

  The external I / F unit 34 is bidirectionally connected to the control unit 32 via a control signal line. The control unit 32 is also connected to the CCD image pickup device 16, the amplifier circuit 18, and A / D converter via the control signal line. The circuit 20, the in-focus detection unit 28, the motor 30, and the image high-resolution unit 26 are bidirectionally connected. Therefore, the user can set shooting conditions such as ISO sensitivity via the external I / F unit 34.

  The output from the image resolution increasing unit 26 is input to the in-focus detection unit 28 through the video signal line, and the in-focus detection unit 28 controls the motor 30 through the control line, thereby controlling the lens system. 12 is adjusted at the in-focus position.

  FIG. 2A is a diagram showing an operation flowchart of the digital camera 10.

  That is, the user enters the pre-shooting mode by setting the shooting condition and then pressing the release button halfway (step S10). During the pre-photographing mode, the CCD image sensor 16 captures a plurality of images, and the captured video signal is converted into a digital signal by the A / D conversion circuit 20 as described above, and the buffer 22 is stored. Then, it is transferred to the high resolution area selection unit 24.

  The high resolution area selection unit 24 selects an image area to be subjected to the high resolution process of the image (step S12). As a means for selecting a region where the resolution of the image is increased, the purpose is to select a region for adjusting the focal position of the image, and therefore it is preferable that the image includes a high-frequency component. The edge area to be selected is mainly selected. In other words, a region having a lot of edge energy, which is an integrated value after convolution of a differential filter such as a Laplacian filter, is preferentially selected as the edge energy. In addition, since there is a high possibility that the main subject to be focused is located in the center of the captured image, a method of selecting the area in the center with priority over the end area of the captured image is also used. Can be used. In addition, when a plurality of images are taken during pre-photographing, if the object is a moving object such as a person, the movement of the subject can be detected by detecting the difference between the images of the plurality of images. May be preferentially selected as a selection region. As described above, the region to be selected is not limited to one place in the image, and may be selected at a plurality of places. The above-described method for selecting a higher resolution area is exemplary, and another method for selecting a higher resolution area may be employed.

  As described above, using the selection area information selected by the high resolution area selection unit 24, only a selected area of each of the plurality of pre-photographed images is trimmed, and the image resolution resolution unit 26 is connected via a video signal line. Forwarded.

  The image high resolution unit 26 performs high resolution processing based on the selection area information selected as described above (step S14). Details of this high resolution processing will be described later. The high resolution image output from the image high resolution unit 26 is input to the in-focus detection unit 28. The in-focus detection unit 28 detects the in-focus using the high resolution image, and controls the motor 30 according to the detection result to adjust the focus of the lens system 12 (step S16). Details of the focus adjustment will be described later. And it is discriminate | determined whether the in-focus state was acquired by this focus adjustment (step S18). This determination method will also be described later. If it is not yet in focus, it is further confirmed whether or not the release button is half-pressed (step S20). Return to and repeat the above process. When the half-press of the release button is released, the operation is terminated.

  Thus, when it is determined in step S18 that the in-focus state has been obtained, a standby state is maintained until the release button is fully pressed, and so-called focus locking is performed. That is, it is determined whether or not the release button is fully pressed (step S22). If the release button is not fully pressed, it is further determined whether or not the release button is half-pressed (step S24). If the half-press is not released, the process returns to step S22. Thus, until the release button is fully pressed, a standby state is established by going through the loop of step S22 and step S24. When it is determined that the in-focus state is obtained in step S18, it is desirable to notify the user by sound or display. In step S24, when the half-press of the release button is released, the operation is stopped.

  If it is detected in step S22 that the release button has been fully pressed, actual shooting is performed with the focus adjusted as described above (step S26). The actually photographed image is transferred from the buffer 22 to the image recording unit 36, where it is recorded (step S28). If necessary, the image recorded in the image recording unit 36 is transferred to the output unit 38 and the digital camera 10 outputs the image. Further, the image recorded in the image recording unit 36 is transferred to the image high resolution unit 26 and used as a resource for image high resolution processing, and the resulting high resolution image is recorded in the image recording unit 36. Alternatively, it can be transferred to the output unit 38 and output from the digital camera 10.

  Needless to say, the high-resolution image obtained by the high-resolution processing in step S14 can be recorded in the image recording unit 36 or output by the output unit 38.

  Next, the image resolution enhancement processing executed in step 14 will be described in detail.

  As described above, the image resolution increasing unit 26 includes the used number control unit 26A, the reference image selecting unit 26B, the image displacement estimating unit 26C, the resolution increasing target image selecting unit 26D, and the resolution increasing processing unit 26E. ing.

  The used number control unit 26A controls the number of used images to be subjected to high resolution processing in accordance with at least one of the photographing mode, focal length, and image magnification specified by the external I / F unit 34. For example, the number-of-use control unit 26A increases the number of images because the amount of light is small in the night scene shooting mode, increases the number when the image magnification is low, and increases the number when the focal length is short. Control becomes possible. It is assumed that the used number control unit 26A selects one or more images to be used for higher resolution including an image serving as a reference for higher resolution prior to shooting.

  FIG. 2B is a flowchart of the image resolution enhancement process. That is, one reference image serving as a reference for image displacement estimation is selected by the reference image selection unit 26B from the plurality of pre-captured images input to the image resolution increasing unit 26 (trimmed) (step) S141). Thereafter, the image displacement estimation unit 26C obtains an image corresponding position between the reference image selected by the reference image selection unit 26B and a pre-photographed image other than the reference image (step S142). Subsequently, a resolution enhancement target image to be subjected to resolution enhancement processing is selected (step S143). Finally, high resolution processing is performed on the high resolution target image (step S144).

  Details of an image displacement estimation algorithm for obtaining a pixel-corresponding position will be described below as an image displacement estimation means performed by the image displacement estimation unit 26C. FIG. 3 is a flowchart showing the processing of the image displacement estimation unit 26C that executes this image displacement estimation algorithm.

  First, one reference image serving as a reference for image displacement estimation selected in step S141 is read (step S1421). Next, the read reference image is deformed with a plurality of image displacements to generate an image sequence (step S1422). For example, the deformation is, for example, 19 (27 out of 27) reference images with a movement parameter of ± 1 pixel in the horizontal and vertical directions and a movement parameter of ± 1 ° [degree] in the rotation direction. 8 types ((1 [pixel], 1 [pixel], -1 [degree])), (1, -1, 1), (-1, 1, 1), (1, -1, -1), ( -1, 1, -1), (-1, -1, 1), (1, 1, 1), (-1, -1, -1)) can be omitted.

  Subsequently, one pre-captured image that performs image displacement estimation with the read standard image is read as a reference image (step S1423). Next, a rough pixel position is associated between the read standard image and reference image by a pixel matching method such as region-based matching (step S1424). Next, a similarity value between the image sequence generated in step S1422 and the reference image is calculated (step S1425). Then, a discrete similarity map is created using the relationship between the image displacement parameter and the similarity value calculated in step S1425 (step S1426). Such a similarity map is schematically shown in FIG. In FIG. 4, the vertical axis represents the similarity, and the smaller the value, the higher the similarity. The abscissa represents the motion parameter obtained by deforming the image.

  Subsequently, an interpolated continuous similarity map is created by interpolating the discrete similarity map created in step S1426, the extreme values of this continuous similarity map are searched, and continuous An extreme value (image displacement estimated value) of the similar similarity value is obtained (step S1427). As a method for interpolating a discrete similarity map into a continuous similarity map in this way, there are parabolic fitting, spline interpolation, and the like. In FIG. 4, a parabola-fitted continuous similarity map is represented as a curve.

  Thereafter, it is confirmed whether or not image displacement estimation has been performed for all target reference images (step S1428). If there is a reference image for which image displacement estimation has not yet been performed, the frame number of the next reference image is determined. Is selected (step S1429), the process returns to step S1423, and the above process is continued.

  If image displacement estimation has been performed for all target reference images (step S1428), the image displacement estimation processing is terminated.

  If the image displacement estimation value is obtained by the image displacement estimation unit 26C in this way, the high resolution target image selection unit 26D then selects the high resolution target image in step S143, and the process proceeds to step S144. By using the estimated image displacement value, the plurality of pre-photographed pre-photographed images, and the selection information of the high-resolution target image, the high-resolution processing unit 26E increases the resolution of the high-resolution target image. Processing is performed.

  FIG. 5 is a flowchart showing the high resolution processing performed in the high resolution processing unit 26E in step S144. In the present embodiment, “super-resolution processing” is used as an algorithm for the resolution enhancement processing. The details of this super-resolution processing are described in “Tanaka / Okutomi, Reconstruction-type Super-resolution Processing Speedup Algorithm, Computer Vision and Image Media (CVIM) Vol. 2004, No. 113, pp. 97-104 (2004). -11) ".

  First, a plurality of low-resolution pre-photographed images are read for use in high-resolution image estimation (step S1441). Subsequently, the pre-photographed image selected by the high-resolution target image selection unit 26D out of the read plural pre-photographed images is set as a high-resolution target image. An initial resolution-enhanced image is created by performing interpolation processing such as linear interpolation and bicubic interpolation on the target image (step S1442). This interpolation process can be omitted in some cases.

  Then, based on the image displacement information estimated by the image displacement estimation unit 26C, the pixel corresponding position between the target image and the other pre-captured image is clarified (step S1443), and the image corresponding position information is used as a basis. The registration image y is generated by performing an overlay process in a coordinate space with the enlarged image of the target image as a reference. The details of the method of generating the registration image y are disclosed in the above-mentioned document “Speed-up Algorithm for Reconstruction Super-Resolution Processing”. In the superimposition processing in this step, for example, when a position is associated between each pixel value of a plurality of pre-photographed images and the enlarged coordinate of the target image, each pixel value is closest to the enlarged coordinate of the target image. A process is performed on the grid points. The image displacement between the target image and the other pre-captured images is the inter-image displacement between the reference image and the target image and the image between the reference image and an image other than the target image estimated by the image displacement estimation unit 26C. Generate by integrating displacements.

  Next, a point spread function (PSF) taking into consideration imaging characteristics such as an optical transfer function (OTF) and a CCD aperture is obtained (step S1444). For PSF, for example, a Gauss function can be easily used.

Next, the evaluation function f (z) is minimized based on the pixel correspondence information in step S1443 and the PSF information in step S1444 (step S1445). This evaluation function f (z) is expressed by the following equation.

Here, y is the registration image generated in step S1443, z is the result image (high resolution image) obtained by increasing the resolution of the target image, and A is an image conversion matrix that represents the imaging system including PSF, CFA, and the like. And includes a point spread function of the optical system, blur due to the sampling aperture, each color component due to the CFA 14, and the like. g (z) is a constraint term in consideration of the smoothness of the image and the correlation between the colors of the image. λ is a weighting factor. For example, the steepest descent method is used to minimize the evaluation function.

When this steepest descent method is used, the differential value of the evaluation function f (z) in the high-resolution image z is calculated, and the differential value is added to the high-resolution image z to update the image. The minimum value of the evaluation function f (z) is obtained. The differential value of the cost value function f (z) is as follows.

This differential value is added to the high-resolution image z, and the image is updated to obtain the minimum value of the evaluation function f (z).

Here, z _n represents the n-th result image (high resolution image) with high resolution, and α represents the step width of the update amount.

  By taking an expression such as Equation (3), the software minimization step is accelerated.

Then, it is determined whether or not the evaluation function f (z) obtained in step S1445 has been minimized (step S1446). Here, if the evaluation function f (z) has not yet been minimized, the high-resolution image z _n is updated (step S1447), and the process returns to step S1445, where the minimum of the evaluation function f (z) is obtained. Perform the process again.

  Thus, if the evaluation function f (z) obtained in step S1445 is minimized, it is determined that the high-resolution image z is obtained, and the high-resolution processing is terminated.

  When a plurality of low resolution images are selected as target images by the high resolution target image selection unit 26D, the processing from step S1442 to step S1446 is performed for all the selected low resolution images.

  FIG. 6 is a block diagram showing an example of the configuration of the resolution enhancement processing unit 26E for performing the resolution enhancement process described above. The high resolution processing unit 26E includes an interpolation enlargement unit 26E1, an image storage unit 26E2, a PSF data holding unit 26E3, a first convolution integration unit 26E4, a registration image generation unit 26E5, an image comparison unit 26E6, and a second convolution integration. A unit 26E7, a regularization term calculation unit 26E8, an update image generation unit 26E9, and a convergence determination unit 26E10.

  Hereinafter, how the resolution enhancement processing described with reference to FIG. 5 is executed by each unit of the resolution enhancement processing unit 26E having such a configuration will be described.

  That is, first, the high resolution target image selected by the high resolution target image selection unit 26D is transferred to the interpolation enlargement unit 26E1, and the interpolation enlargement unit 26E1 performs interpolation enlargement of the target image (described above). Corresponding to step S1441). Examples of the interpolation enlargement method used include bilinear interpolation and bicubic interpolation. The interpolated and enlarged image is transferred to the image storage unit 26E2 and stored therein.

The image stored in the image storage unit 26E2 is transferred to the first convolution integration unit 26E4, and the first convolution integration unit 26E4 convolves with the PSF data transferred from the PSF data holding unit 26E3. Integration is performed (this convolution integral corresponds to the Az portion in the calculation of A ^T (y−Az) in the above equation (2)).

  The target image is also transferred to the registration image generation unit 26E5 together with a plurality of other pre-captured images, and the registration image generation unit 26E5 uses the inter-frame obtained by the image displacement estimation unit 26C. Based on the image displacement information, a registration process is generated by performing superimposition processing in a coordinate space based on the enlarged coordinates of the target image (corresponding to step S1443). For example, when the position is associated between each pixel value of the plurality of images and the enlarged coordinates of the target image, the superimposing process is performed on the lattice point closest to the enlarged coordinates of the target image. This is done by the process of placing pixels. At this time, it is assumed that a plurality of pixel values are placed on the same grid point, but in this case, an average value taken between the plurality of pixel values can be placed.

Next, the image data subjected to the convolution operation by the first convolution integration unit 26E4 is transferred to the image comparison unit 26E6, and the registration image generated by the registration image generation unit 26E5 is transmitted to the image comparison unit 26E6. , The difference value of the pixel value is calculated at the same pixel position, and difference image data is generated (corresponding to y-Az in the calculation of ^AT (y-Az) in the above formula (2)). . The difference image data generated in the image comparison unit 26E6 is transferred to the second convolution integration unit 26E7, and the second convolution integration unit 26E7 performs convolution with the PSF data provided by the PSF data holding unit 26E3. The integration is performed (corresponding to ^AT · (y−Az) in the calculation of ^AT (y−Az) in the above equation (2)).

  Further, a regularization image is generated by the regularization term calculation unit 26E8 from the image stored in the image storage unit 26E2. That is, in the regularization term calculation unit 26E8, for example, color conversion processing from RGB to YCrCb is performed on the image stored in the image storage unit 26E2, and in the Cr and Cb components (color difference components), A convolution operation process of a frequency wide-pass filter (Laplacian filter) is performed to generate a regularized image. By using the regularized image generated by this method in the updated image generation unit 26E9, the foreseeing information of the image that “the color difference component of the image is generally a smooth change” is used. A stable image can be obtained stably (corresponding to the partial differentiation of g (z) in the above equation (2) by z).

  Then, the image generated by the second convolution integration unit 26E7, the image stored in the image storage unit 26E2, and the image generated by the regularization term calculation unit 26E8 are transferred to the update image generation unit 26E9. The updated image generation unit 26E9 generates an updated image by generating a weighted image of these three images (corresponding to the above equation (3)). The generated update image is transferred to the convergence determination unit 26E10, and the convergence determination is performed in the convergence determination unit 26E10. In this convergence determination, it may be determined that the image update operation has converged when the number of iterations required for convergence is greater than a certain number. An update image may be held, a difference from the current update image may be taken, and if it is determined that the difference is smaller than a certain value, it may be determined that the image update operation has converged. When the convergence determination unit 26E10 determines that the update operation has converged, the updated image is output to the outside as a high-resolution image.

  On the other hand, when it is determined that the update operation has not converged, the update image is transferred to the image storage unit 26E2 and used for generation of the update image next time of the repeated update operation. The Then, for the next update image generation, the first convolution integration unit 26E4, the regularization term calculation unit 26E8, and the update image generation unit 26E9 are provided.

  By repeating the above processing, the update image generated by the update image generation unit 26E9 is updated, and a final high-resolution image is obtained.

  FIG. 7 is a diagram showing a comparison between the final high-resolution image obtained in this way and the area selection image (low-resolution image) selected by the high-resolution area selection unit 24. In the figure, the right side is a low-resolution image before the resolution is increased, and the left side is a high-resolution image that has been increased based on the low-resolution image. The upper side has high sharpness, and the lower side has low sharpness. As a property of the super-resolution processing algorithm employed in the present embodiment, as a result of the high resolution processing, an image with low sharpness is less effective for an image with low sharpness, and an image with high sharpness is obtained. Therefore, it has the property that the resolution restoration effect is high. In the example shown in FIG. 7, the high-resolution image with high sharpness has a higher resolution effect on the corresponding low-resolution image than the high-resolution image with low sharpness. The nature of the algorithm itself favors the control for focusing described below.

  As described above, the resolution enhancement process in the image resolution enhancement unit 26 increases the resolution of the selected areas of the plurality of images selected by the resolution enhancement area selection unit 24. Thus, the high resolution image obtained by the image high resolution unit 26 is transferred to the in-focus detection unit 28.

  In step S16, the in-focus detection unit 28 controls the motor 30 by performing a DFF (Depth From Focus) process represented by FIG. Perform alignment. That is, by convolving a high-pass filter such as a Laplacian filter into the image, the edge sharpness of the high-resolution image is detected, the motor 30 is controlled in the direction of increasing the sharpness, and the focus position Align. As described above with reference to FIG. 7, the sensitivity of the edge sharpness is improved by performing the high resolution processing due to the characteristics of the high resolution processing algorithm employed in the present embodiment. This is effective, and the accuracy of focusing is improved.

  When the focus is thus aligned, a plurality of images are again captured by the CCD image sensor 16, and the pre-captured image is passed through the amplification circuit 18, the A / D conversion circuit 20, and the buffer 22 as described above. Then, the image is transferred to the high resolution area selection unit 24, the area is selected, transferred to the image high resolution unit 26, the resolution of the selected area is increased, and the resulting high resolution image is transferred to the in-focus detection unit 28. The sharpness of the high-resolution image is detected, the motor 30 is controlled, and the focus is aligned.

  The above processing is repeated, and when the sharpness reaches the maximum value, the focus detection unit 28 determines that the focus position of the lens system 12 is in focus.

  In the pre-photographing mode, after it is determined that the focal position is in focus, the user fully presses the release button, so that the digital camera 10 enters the main photographing mode and performs the main photographing of the subject. The actually captured image is recorded in the image recording unit 36 from the buffer 22 via the video signal line. The recorded image is transferred to the image resolution increasing unit 26 via the video signal line as necessary, and is used as a resource for the image high resolution processing, and also to the output unit 38 via the video signal line. The image is output from the digital camera 10.

  The focus adjustment using the high resolution processing in the case where the high resolution algorithm (super-resolution processing algorithm) shown in FIG. 5 is employed has been described above. Of course, the present invention is not limited to this super-resolution processing algorithm, and other algorithms may be adopted.

[Second Embodiment]
FIG. 9 is a block diagram schematically showing the configuration of the digital camera 10 as the imaging apparatus according to the second embodiment of the present invention. The digital camera 10 according to the present embodiment has the same configuration as that of the first embodiment, and is further connected to the buffer 22, the high resolution area selection unit 24, and the image high resolution unit 26, and is output from them. An image display unit 40 such as an LCD for displaying a video signal is provided. The external I / F unit 34 is provided with a focusing mechanism 34A.

  FIG. 10 is a diagram showing an operation flowchart of the digital camera 10 in the second embodiment. In the present embodiment, focus detection is performed by selecting either autofocus or manual focus.

  That is, when the user presses the release button halfway after setting the shooting conditions, first, it is determined whether or not the manual focus mode is selected (step S30), that is, the manual focus mode is not selected. If it is determined that the autofocus mode is selected, autofocus processing is executed (step S32). For this autofocus process, the processes in steps S10 to S28 in the first embodiment can be applied as they are, and therefore the description thereof is omitted.

  On the other hand, if it is determined in step S30 that the manual focus mode is selected, the pre-shooting mode is entered (step S34). In this pre-photographing mode, a plurality of pre-photographed images are photographed and transferred to the high resolution area selection unit 24 via the buffer 22 in the same manner as in the process of step S10.

  The high resolution area selection unit 24 selects an area of the image to be subjected to the high resolution process of the image (step S36). Since the high resolution area selection process is the same as that in step S12, detailed description thereof is omitted.

  Then, using the selection area information selected by the high resolution area selection unit 24, only a selected area of the plurality of pre-photographed images is subjected to trimming processing, and the image high resolution unit 26 is connected via a video signal line. The image resolution increasing unit 26 performs a resolution increasing process based on the selection area information (step S38). Since this high resolution processing is the same processing as step S14, detailed description thereof will be omitted.

  The high resolution image output from the image high resolution unit 26 is input to the image display unit 40, and the high resolution image is displayed on the image display unit 40 (step S40).

  The user visually checks the high-resolution image displayed on the image display unit 40 to determine whether the image is in focus. When the user determines that the image is in focus, the user presses the release button fully, Real shooting can be performed. That is, after the high resolution image is displayed in step S40, it is determined whether or not the release button has been fully pressed (step S42). If it is determined that the release button has been fully pressed, the main shooting mode is entered. Then, actual photographing is performed (step S44), and the photographed image data is recorded in the image recording unit 36 (step S46). If necessary, the image recorded in the image recording unit 36 is transferred to the output unit 38 and the digital camera 10 outputs the image. Further, the image recorded in the image recording unit 36 is transferred to the image high resolution unit 26 and used as a resource for image high resolution processing, and the resulting high resolution image is recorded in the image recording unit 36. Or output from the digital camera 10 or displayed on the image display unit 40.

  On the other hand, when the user visually observes the high resolution image displayed on the image display unit 40 and determines that the image is not in focus, the manual focus procedure can be performed without fully pressing the release button. it can. In this manual focus, the user operates the focusing mechanism 34A provided in the external I / F unit 34 to control the motor 30 via the control unit 32 and adjust the focus of the lens system 12. It is.

  Therefore, when it is determined in step S42 that the release button is not fully pressed, it is determined whether or not the user has operated the focusing mechanism 34A (step S48). If it is determined that the focus adjustment mechanism 34A is not operated, it is further determined whether or not the release button is half-pressed (step S50). If it is determined that the half-press of the release button has been completed, the shooting is stopped. Conversely, if it is determined that the half-press of the release button has been continued, the process returns to step S48.

  As described above, the process is repeated as long as the user presses the release button halfway without operating the focusing mechanism 34A, and forms a standby state until the user fully presses the release button or operates the focusing mechanism 34A. To do.

  If it is determined in step S48 that the user has operated the focusing mechanism 34A, the motor 30 is controlled via the control unit 32 to adjust the focus of the lens system 12 according to the operation. (Step S52). Thereafter, it is determined whether or not the user continues to press the release button halfway (step S54). If the release button is pressed halfway, the process returns to step S34, and the focus adjusted by the user is used. The operations for manual focus adjustment in steps S34 to S52 are repeated. On the other hand, if it is determined in step S54 that the half-press of the release button has been completed, shooting is terminated.

  As a result of repeating the operation for manual focus adjustment as described above and the user repeatedly performing focusing using the focusing mechanism 34A, the user visually observes the high-resolution image displayed on the image display unit 40, When it is determined that the subject is in focus, the user can press the release button fully (branch to YES in step S42) and perform the actual photographing.

  Note that the external I / F unit 34 of the digital camera 10 according to the second embodiment is configured to allow the user to select either manual focus or autofocus shooting mode. This shooting mode can be selected by switching means such as a switch provided in the external I / F unit 34. The user presses the release button halfway before setting the switching means to manual focus or autofocus before entering the above flow for focusing by pressing the release button halfway. Of course, the user can switch between manual focus and auto focus during the in-focus flow by switching the switching means while the release button is half-pressed.

  Note that the display of the high-resolution image on the image display unit 40 is performed, for example, as shown in FIG.

  That is, as shown in FIG. 11A, the image display section 40 of the digital camera body 10A in the second embodiment is configured as a liquid crystal display panel 40A provided on the back surface of the digital camera body 10A. Then, a part of the image display area 40B of the liquid crystal display panel 40A is set as a high resolution image display area 40C, and an image subjected to the high resolution processing is displayed here. In addition to the high-resolution image display area 40C of the image display area 40B, information on parameters related to shooting such as an arbitrary image, shutter speed, ISO sensitivity, focal length, and the like can be displayed. The captured image at the current time to be transferred may be displayed. Further, FIG. 11A shows as an example that the digital camera body 10A includes a power switch 34B, a release button 34C, and an operation button 34D as part of the external I / F unit 34. .

  FIGS. 11B and 11C show display examples of the high-resolution image on the image display unit 40. As shown in these drawings, a part of the image display area 40B is used as a high-resolution image display area 40C, and a high-resolution image generated from a plurality of pre-photographed images is displayed there, so that In addition, visual focusing can be easily performed with an image having a high resolution, compared to the case where an enlarged image of one image is used for image display. In addition, when the super-resolution processing algorithm is used as the high resolution processing algorithm, the sharpness drop between the high sharpness and the low sharpness becomes large, and it is confirmed whether the focal position is in alignment. It will be easy.

  That is, when the sharpness of the displayed high-resolution image is low as shown in FIG. 11C, the user operates the focusing mechanism 34A to increase the height as shown in FIG. The focus of the digital camera 10 is adjusted so that the sharpness of the resolution image is increased. As a result, when the high-resolution image displayed in the high-resolution image display area 40C becomes an image with high sharpness as illustrated in FIG. When it is determined that the subject is in focus and the release button 34C is fully pressed, the digital camera 10 can be used for the main shooting mode to perform the main shooting of the subject. By such an operation, the user can take an image with an in-focus position. As described with reference to FIG. 7 for the first embodiment, as a property of the algorithm of the super-resolution processing, as a result of the high-resolution processing, a resolution restoration effect is obtained for an image with low sharpness. There is a characteristic that the effect of restoring the sense of resolution is high for an image with a small amount of sharpness. As a result of this property, there is an advantageous effect that the sensitivity of the edge sharpness is improved, and the sensitivity of the focus alignment is improved.

  FIG. 12A is a diagram showing another display method, in which the high resolution image display area 40C and the low resolution image display area 40D are arranged in the image display area 40B of the liquid crystal display panel 40A to increase the resolution. The processed image and the low resolution area selection image selected by the high resolution area selection unit 24 are displayed side by side. In this case, the region selection image is enlarged and displayed to the same size as the high resolution image by the bilinear method or the like in order to compare the image with the high resolution image.

  According to this display method, the image used for image display is a high-resolution image generated from a plurality of pre-captured images and an enlarged region selection image, so that the high-resolution image has improved sharpness. It becomes clear that In particular, when the super-resolution processing algorithm is used as the high-resolution processing algorithm, two images are sharper than when only an enlarged image of one image is used for image display as in the conventional case. The difference in sharpness between the case where the degree is high and the case where the degree is low becomes clear. By comparing the high-resolution image and the low-resolution image, it is possible to easily confirm whether or not the focus position is correct. In addition, as a property of this super-resolution processing algorithm, as a result of high resolution processing, there is little effect of restoring resolution for images with low sharpness, and resolution for images with high sharpness. It has the property that the feeling restoration effect is high. An image between a high-resolution image generated using a plurality of images displayed in the high-resolution image display area 40C and a low-resolution image composed of one image displayed in the low-resolution image display area 40D. By comparing the feeling of resolution, it is possible to check whether the focus position is correct by taking advantage of the characteristic that the sharpness of the resolution is large if the sharpness is high and the sharpness of the resolution is small if the sharpness is low. Can be made easier. In addition to the high-resolution image display area 40C and the low-resolution image display area 40D in the image display area 40B, information on parameters related to shooting such as an arbitrary image, shutter speed, ISO sensitivity, and focal length can be displayed. However, the captured image at the current time transferred to the buffer 22 may be displayed.

  12B and 12C show display examples on the image display unit 40. FIG. The high-resolution image shown in FIG. 12B has high sharpness, and the high-resolution image shown in FIG. 12C has low sharpness. As described above, when the user views the high-resolution image displayed in the high-resolution image display area 40C and determines that the sharpness of the image is low, the user uses the focusing mechanism 34A to focus. Perform position alignment. Further, in this display method, since a low-resolution area selection image is displayed in the low-resolution image display area 40D, when the above-described image resolution is compared, if the sharpness is high, the resolution drop If the image quality is large and the sharpness is low, it is possible to make it easier to check whether or not the focal position is correct by taking advantage of the characteristics of the super-resolution processing that the drop in resolution is small. As a result, when the high-resolution image displayed in the high-resolution image display area 40C becomes an image with high sharpness as illustrated in FIG. When it is determined that the subject is in focus and the release button 34C is fully pressed, the digital camera 10 is set to the main shooting mode to perform the main shooting of the subject. By such an operation, the user can take an image with an in-focus position.

  Further, a display method as shown in FIGS. 13A, 13B, or 13C may be employed.

  In the display method shown in FIG. 13A, two high-resolution image display areas 40C1 and 40C2 are arranged in the image display area 40B, and the first high-resolution image display area 40C1 is captured by the previous focus adjustment. The resolution-enhanced image generated from the plurality of images is displayed, and in the second resolution-enhanced image display area 40C2, one of the resolution-enhanced images displayed in the first resolution-enhanced image display area 40C1 is displayed. The high resolution image with the focus adjusted before the rotation is displayed. In the display method shown in FIG. 11 (A), only one high-resolution image generated from a plurality of images taken by the previous focus adjustment is displayed. In order to make it easier to determine the focus, the high-resolution images with the focus adjusted once before are displayed side by side. In particular, as a property of the super-resolution processing, as a result of the high resolution processing, the effect of restoring resolution is small for images with low sharpness, and the resolution for images with high sharpness is low. It has the property of high restoration effect. Therefore, by displaying side by side as in the present display method, it is possible to more easily compare the resolution restoration effect of the two images.

  Further, in this display method, the explanation display 40E (that is, the caption) of “before adjustment” and “after adjustment” is respectively displayed so that it can be understood which resolution enhancement image is the resolution enhancement image generated by the previous focus adjustment. It is preferable to display in the vicinity of the first and second high resolution image display areas. The display position of the comment display 40E and whether or not to display the comment display 40E may be selectable by the user via the external I / F unit 34. Further, the explanation display 40E is not limited to “before adjustment” and “after adjustment” as shown in the figure, but may be other things. Furthermore, the display positions of the first high-resolution image display area 40C1 and the second high-resolution image display area 40C2 may be adjustable by the user via the external I / F unit 34. In addition to the first high-resolution image display area 40C1 and the second high-resolution image display area 40C2 in the image display area 40B, any image, shutter speed, ISO sensitivity, focal length, and the like can be captured. Information on the parameters involved can be displayed, but in particular, a captured image at the current time transferred to the buffer 22 may be displayed.

  In addition, in the display method shown in FIG. 13B, in the image display area 40B, an enlarged display of the high resolution image by the high resolution image display area 40C and the low resolution area selection image by the low resolution image display area 40D. Are displayed alternately. In the display method shown in FIG. 12A, these two display areas are displayed side by side. In this display method, information is displayed in an area outside the display area by alternately displaying the two display areas. In particular, when a captured image at the current time transferred to the buffer 22 is displayed, there is an advantage that information outside these display areas can be easily confirmed. Also, as a property of the super-resolution processing algorithm, as a result of high resolution processing, there is little resolution restoration effect for images with low sharpness, and resolution restoration effect for images with high sharpness. There is a nature that is high. An image between a high-resolution image generated using a plurality of images displayed in the high-resolution image display area 40C and a low-resolution image composed of one image displayed in the low-resolution image display area 40D. By comparing the feeling of resolution, it is possible to check whether the focus position is correct by taking advantage of the characteristic that the sharpness of the resolution is large if the sharpness is high and the sharpness of the resolution is small if the sharpness is low. It is possible to display the two images alternately by this display method.

  The user can adjust the display times of the high-resolution image display area 40C and the low-resolution image display area 40D via the external I / F unit 34, and set each image display area to a different location. Can do. In addition, a time interval in which no image display region is displayed can be set between the display changes of the high-resolution image display region 40C and the low-resolution image display region 40D, and the time interval can be adjusted. In addition to the high-resolution image display area 40C and the low-resolution image display area 40D in the image display area 40B, information on parameters related to shooting such as an arbitrary image, shutter speed, ISO sensitivity, and focal length can be displayed. However, in particular, the captured image at the current time transferred to the buffer 22 as described above may be displayed.

  In the display method shown in FIG. 13C, two high-resolution image display areas 40C1 and 40C2 and two low-resolution image display areas 40D1 and 40D2 are arranged in the image display area 40B. The first high-resolution image display area 40C1 displays a high-resolution image generated from a plurality of images taken by the previous focus adjustment, and the second high-resolution image display area 40C2 includes The high-resolution image with the focus adjusted once before the high-resolution image displayed in the first high-resolution image display area 40C1 is displayed. The first low-resolution image display area 40D1 displays a low-resolution area selection image captured with the focus of the high-resolution image displayed in the first high-resolution image display area 40C1. In the second low-resolution image display area 40D2, a low-resolution area selection image captured with the focus of the high-resolution image displayed in the second high-resolution image display area 40C2 is displayed. Here, the display areas are arranged so that the high resolution image and the area selection image generated with the same focus are displayed side by side. Further, in the vicinity of the first and second high resolution image display areas 40C1 and 40C2, explanation display 40E of “after adjustment” and “before adjustment” is performed, respectively.

  As the nature of the super-resolution processing algorithm, as a result of the high resolution processing, the image restoration effect is low for images with low sharpness, and the image restoration effect is high for images with high sharpness. By displaying the images side by side as in the present display method, it is possible to more easily compare the resolution restoration effect of the two images. Further, a low resolution composed of a high resolution image generated using a plurality of images displayed in the high resolution image display areas 40C1 and 40C2 and a single image displayed in the low resolution image display areas 40D1 and 40D2. By comparing the image resolution between images, it is possible to take advantage of the characteristic that the sharpness of the resolution is large if the sharpness is high and the resolution is small if the sharpness is low. It is possible to check whether the position is correct. In this display method, for example, when a captured image at the current time to be transferred to the buffer 22 is displayed on the background, there is a possibility that it is difficult to confirm background information and images because there are four image display areas. There is. Therefore, the user can check background information and images by temporarily deleting any or all of the four image display areas via the external I / F unit 34. It is also possible to change the display positions of the four image display areas. Further, the explanation display 40E does not necessarily need to be “before adjustment” and “after adjustment”, and may be another display, and the user can cancel these displays via the external I / F unit 34. . In addition to the high-resolution image display area 40C and the low-resolution image display area 40D in the image display area 40B, information on parameters related to shooting such as an arbitrary image, shutter speed, ISO sensitivity, and focal length can be displayed. However, in particular, the captured image at the current time transferred to the buffer 22 as described above may be displayed.

  The display method described with reference to FIGS. 11A, 12A, 13A, 13B, and 13C is selected by the user via the external I / F unit. You may comprise.

  The present invention has been described above based on the embodiments. However, the present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the gist of the present invention. is there.

  In particular, the above functions can be realized by supplying a software program that implements the functions of the above-described embodiments to a computer and executing the program by the computer.

(Appendix)
The invention having the following configuration can be extracted from the specific embodiment.

(1) In an imaging device that electronically records an image of a subject,
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
A focus detection means for detecting a focus position of the imaging optical system, using an image whose resolution is increased by the image resolution enhancement means;
An adjustment unit that causes the focusing unit to adjust the focus of the imaging optical system based on the in-focus position detected by the focus detection unit;
An imaging apparatus comprising:

(Corresponding embodiment)
The first and second embodiments correspond to the embodiment relating to the imaging device described in (1). In these embodiments, the digital camera 10 is the imaging device, the CCD imaging device 16 is the imaging device, the lens system 12 is the imaging optical system, the motor 30 is the focusing means, and the high-resolution area selection unit. 24 and the image resolution enhancement unit 26 correspond to the image resolution enhancement unit, and the focus detection unit 28 corresponds to the focus detection unit and the adjustment unit.

(Function and effect)
According to the imaging device described in (1), when a single high-resolution image is generated using a plurality of images and the edge sharpness is calculated using this image, the resolution near the edge is determined. Can be improved and noise can be reduced. As a result, the in-focus detection means can calculate the edge sharpness with high accuracy, and the adjustment means adjusts the focusing means based on the edge sharpness, so that focusing (autofocus) is performed with high accuracy. Can do.

(2) In an imaging device that electronically records an image of a subject,
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
Image display means for displaying an image having a high resolution in the image high resolution means;
Adjusting means capable of manually adjusting the focusing means;
An imaging apparatus comprising:

(Corresponding embodiment)
The embodiment relating to the imaging device described in (2) corresponds to the second embodiment. In the embodiment, the digital camera 10 is used as the imaging device, the CCD imaging device 16 is used as the imaging device, the lens system 12 is used as the imaging optical system, and the motor 30 and the control unit 32 are used as the focusing means. The area selection unit 24 and the image resolution enhancement unit 26 correspond to the image resolution enhancement unit, the image display unit 40 corresponds to the image display unit, and the focusing mechanism 34A and the control unit 32 correspond to the adjustment unit.

(Function and effect)
According to the imaging apparatus described in (2), since a single high-resolution image is generated using a plurality of images and the image is displayed on the image display unit, the user uses the adjustment unit. When visual focusing is performed, focusing can be performed using an image with improved resolution, and focusing can be performed with high accuracy.

(3) a focal point detection unit for detecting a focal point position of the imaging optical system using the image whose resolution has been increased by the image high resolution unit;
Mode setting means for switching between auto focus mode and manual focus mode;
Further comprising
The adjusting unit causes the focusing unit to adjust the focus of the imaging optical system based on the focus position detected by the focus detection unit when the autofocus mode is set by the mode setting unit. The imaging apparatus according to claim 2.

(Corresponding embodiment)
The embodiment relating to the imaging device described in (3) corresponds to the second embodiment. In the embodiment, the in-focus detection unit 28 corresponds to the in-focus detection unit, and the external I / F unit 34 corresponds to the mode setting unit.

(Function and effect)
According to the image pickup apparatus described in (3), when the user is not satisfied with the focusing by the manual focus, the focusing by the autofocus can be attempted by switching the mode setting means. In addition, autofocus mode can be selected from the beginning and focusing can be performed by autofocus.

  (4) The image high-resolution means is a super-resolution process for generating a high-resolution image using a plurality of images of the subject imaged by the imaging device. The imaging device according to any one of the above.

(Corresponding embodiment)
The embodiment relating to the imaging device described in (4) corresponds to the first and second embodiments.

(Function and effect)
According to the image pickup apparatus described in (4), as a property of the super-resolution processing algorithm, as a result of the high resolution processing, the effect of restoring resolution is small for an image with low sharpness, and the sharpness is low. Due to the nature of high resolution restoration effect for high images, the difference in edge sharpness is not obvious between low resolution images that have not undergone high resolution processing when autofocusing. However, the difference in the edge sharpness of the image subjected to the high resolution processing is remarkably increased, and the in-focus detection by the in-focus detection means becomes easy. Also, as the nature of this super-resolution processing algorithm, if the sharpness is high, the difference in resolution between the high-resolution image and the low-resolution image is large, and if the sharpness is low, these resolutions Because of the small head drop, it is possible to focus on a focal point where sharpness increases by visually observing the restoration effect of the resolution of the high-resolution image when performing manual focus. It is.

  (5) The imaging apparatus according to any one of claims 1 to 4, wherein the image high-resolution means includes high-resolution area selection means for selecting an image area to be subjected to image high-resolution processing.

(Corresponding embodiment)
The embodiment relating to the imaging device described in (5) corresponds to the first and second embodiments. In those embodiments, the resolution enhancement area selection unit 24 corresponds to the resolution enhancement area selection unit.

(Function and effect)
According to the imaging device described in (5), since it is preferable that the image includes a high-frequency component, the main subject to be selected for the edge region included in the image or focused is the captured image. By selecting an appropriate image for focusing, such as performing area selection with priority on the center area over the edge area of the captured image. Focusing accuracy can be increased and focusing can be facilitated.

  (6) The image display means displays the high-resolution image, and at the same time enlarges and displays the image before the high-resolution image is increased in resolution. The imaging device according to any one of 2 to 5.

(Corresponding embodiment)
The embodiment relating to the imaging device described in (6) corresponds to the second embodiment.

(Function and effect)
According to the imaging device described in (6), it is apparent that the edge sharpness is improved in the high-resolution image by displaying the high-resolution image and the low-resolution image side by side. In addition, when the super-resolution processing algorithm is used for high resolution processing, the resolution restoration effect is small for images with low sharpness, and the resolution restoration effect for images with high sharpness. The sharpness can be improved by comparing the resolution of the image between the high-resolution image generated using a plurality of images and the low-resolution image composed of one image. It is possible to easily check whether the focal position is correct by taking advantage of the characteristic that if the height is high, the resolution drop is large, and if the sharpness is low, the resolution drop is small.

  (7) The display means alternately displays the high-resolution image and an enlarged image of the high-resolution image before the high-resolution image is increased. Item 6. The imaging device according to any one of Items 2 to 5.

(Corresponding embodiment)
The second embodiment corresponds to the embodiment related to the imaging device described in (7).

(Function and effect)
According to the imaging device described in (7), since the high resolution image and the low resolution image are alternately displayed, the effect of the high resolution processing becomes clear. Further, by alternately displaying the information, when information is displayed in an area outside the display area of the high resolution image and the low resolution image, there is an advantage that it is easy to confirm the information. In addition, when the super-resolution processing algorithm is used as the high resolution processing, the resolution recovery effect is small for images with low sharpness and high sharpness as a result of the high resolution processing. The resolution restoration effect is high, and the resolution of the image is compared between a high resolution image generated using multiple images and a low resolution image consisting of a single image. By doing so, it is possible to make it easy to check whether the focus position is correct by taking advantage of the characteristic that the sharpness is high when the sharpness is high and the sharpness is low when the sharpness is low. .

  (8) The imaging apparatus according to any one of claims 2 to 5, wherein the display unit displays the high-resolution images before and after adjustment by the adjustment unit side by side.

(Corresponding embodiment)
The second embodiment corresponds to the embodiment related to the imaging device described in (8).

(Function and effect)
According to the imaging device described in (8), focusing is facilitated by comparing high resolution images with improved resolution near the edge by high resolution processing. Also, when the super-resolution processing algorithm is adopted as the resolution enhancement processing, the resolution restoration effect is small for images with low sharpness as a result of the resolution enhancement processing, and for images with high sharpness. Has a high resolution restoration effect, and by displaying two high-resolution images side by side, the resolution restoration effect of the two images can be more easily compared.

  (9) The display means displays the high-resolution image before and after adjustment by the adjustment means and the enlarged image of the image before the high-resolution image before the high-resolution image is displayed. The imaging device according to claim 2, wherein the imaging device is an imaging device.

(Corresponding embodiment)
The second embodiment corresponds to the embodiment related to the imaging device described in (9).

(Function and effect)
According to the imaging device described in (9), focusing is facilitated by comparing high resolution images with improved resolution near the edge by high resolution processing. Further, it becomes clear that the high-resolution image has high sharpness by displaying the high-resolution image and the low-resolution image side by side. Furthermore, when the algorithm of super-resolution processing is adopted as the resolution enhancement processing, the resolution restoration effect is small for images with low sharpness as a result of the resolution enhancement processing, and for images with high sharpness. Has a high resolution restoration effect, and by displaying two high-resolution images side by side, the resolution restoration effect of the two images can be more easily compared. Then, by comparing the resolution of the image between the high-resolution image generated using a plurality of images and the low-resolution image composed of one image, if the sharpness is high, the resolution It is possible to make it easy to confirm whether or not the focus position is correct by taking advantage of the characteristics of this algorithm that the drop of the resolution is small if the drop is large and the sharpness is low.

  (10) The image high-resolution means includes means for controlling the number of original images to be subjected to the high-resolution processing according to at least one of a shooting mode, a focal length, and an image magnification. The imaging device according to claim 1.

(Corresponding embodiment)
The embodiment relating to the imaging device described in (10) corresponds to the first and second embodiments. In those embodiments, the used number control unit 26A corresponds to means for controlling the number of original images.

(Function and effect)
According to the imaging device described in (10), for example, the number of images is increased because the amount of light is small in the night scene shooting mode, the number is increased when the image magnification is low, or the number is increased when the focal length is short. It is possible to control the number of original images necessary for high resolution processing according to the shooting conditions.

(11) A focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
Using this high-resolution image, a procedure for detecting the in-focus position of the imaging optical system having the action of a lens,
A procedure for adjusting the focus of the imaging optical system based on the detected focal position,
Focus control program to execute.

(Corresponding embodiment)
The embodiment relating to the focus control program described in (11) corresponds to the first and second embodiments. In these embodiments, the digital camera 10 is the imaging device, the CCD imaging device 16 is the imaging device, the lens system 12 is the imaging optical system, and steps S12 and S14 are steps for increasing the resolution. Corresponds to the procedure for detecting the in-focus position and the procedure for adjusting the focus of the imaging optical system, respectively.

(Function and effect)
According to the focus control program described in (11), a single high-resolution image is generated using a plurality of images, and the edge sharpness is calculated using this image when calculating the edge sharpness. The resolution can be improved and noise can be reduced. As a result, the edge sharpness can be calculated with high accuracy in the procedure of detecting the in-focus position, and the focusing (autofocus) can be performed with high accuracy by means of adjusting the focus of the imaging optical system based on this edge sharpness. Can be done.

(12) A focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
The procedure to display this high resolution image,
A procedure for accepting an operation for adjusting the focus of the imaging optical system having the action of a lens;
A procedure for adjusting the focus of the imaging optical system in accordance with the operation;
Focus control program to execute.

(Corresponding embodiment)
The embodiment related to the focus control program described in (12) corresponds to the second embodiment. In this embodiment, the digital camera 10 is the imaging device, the CCD imaging device 16 is the imaging device, the lens system 12 is the imaging optical system, and steps S36 and S38 are steps for increasing the resolution. The procedure for displaying the high-resolution image corresponds to the procedure for accepting an operation for adjusting the focus of the imaging optical system in step S48, and the procedure for adjusting the focus of the imaging optical system, respectively.

(Function and effect)
According to the focus control program described in (12), one resolution-enhanced image is generated using a plurality of images and the image is displayed. Therefore, the user can select an image with improved resolution. Thus, the operation of adjusting the focus of the image pickup optical system can be performed with high accuracy. As a result, the procedure for adjusting the focus of the image pickup optical system can be performed, whereby the focus can be adjusted with high accuracy.

FIG. 1 is a schematic block diagram of a digital camera as an imaging apparatus according to the first embodiment of the present invention. FIG. 2 is an operation flowchart of the digital camera according to the first embodiment. FIG. 3 is a diagram illustrating a flowchart of processing performed in the image displacement estimation unit. FIG. 4 is a diagram illustrating an example of processing using parabolic fitting to interpolate a discrete similarity map into a continuous similarity map. FIG. 5 is a diagram showing a flowchart of the high resolution processing performed in the high resolution processing unit. FIG. 6 is a schematic block diagram showing the configuration of the resolution enhancement processing unit. FIG. 7 is a diagram illustrating an example of how a high resolution effect appears due to a difference in sharpness in a high resolution image. FIG. 8 is a conceptual diagram showing focusing by DFF processing. FIG. 9 is a schematic block diagram of a digital camera as an imaging apparatus according to the second embodiment of the present invention. FIG. 10 is a diagram illustrating an operation flowchart of the digital camera according to the second embodiment. FIG. 11A is a rear view of a digital camera for explaining a display method in the second embodiment, and FIGS. 11B and 11C are diagrams showing display examples in the display method. FIG. 12A is a rear view of a digital camera for explaining another display example, and FIGS. 12B and 12C are diagrams each showing a display example in the display method. FIGS. 13A to 13C are diagrams showing still another display method.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 10 ... Digital camera, 10A ... Digital camera main body, 12 ... Lens system, 14 ... CFA, 16 ... CCD image sensor, 18 ... Amplification circuit, 20 ... A / D conversion circuit, 22 ... Buffer, 24 ... High resolution area selection , 26 ... Image resolution increasing unit, 26A ... Used number control unit, 26B ... Reference image selection unit, 26C ... Image displacement estimation unit, 26D ... High resolution target image selection unit, 26E ... High resolution processing unit, 26E1 Interpolation enlargement unit 26E2 Image storage unit 26E3 PSF data holding unit 26E4 First convolution integration unit 26E5 Registration image generation unit 26E6 Image comparison unit 26E7 Second convolution integration unit 26E8: Regularization term calculation unit, 26E9: Update image generation unit, 26E10: Convergence determination unit, 28 ... Focus detection unit DESCRIPTION OF SYMBOLS 30 ... Motor, 32 ... Control part, 34 ... External I / F part, 34A ... Focusing mechanism, 34B ... Power switch, 34C ... Release button, 34D ... Operation button, 36 ... Image recording part, 38 ... Output part, 40 DESCRIPTION OF SYMBOLS ... Image display part, 40A ... Liquid crystal display panel, 40B ... Image display area, 40C, 40C1, 40C2 ... High resolution image display area, 40D, 40D1, 40D2 ... Low resolution image display area, 40E ... Explanation display.

Claims (12)

In an imaging device that electronically records a subject image,
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
A focus detection means for detecting a focus position of the imaging optical system, using the image whose resolution is increased by the image resolution enhancement means;
An adjustment unit that causes the focusing unit to adjust the focus of the imaging optical system based on the in-focus position detected by the focusing detection unit;
An imaging apparatus comprising: In an imaging device that electronically records a subject image,
An image sensor;
An imaging optical system having the action of a lens;
Focusing means for focusing the imaging optical system;
Image high-resolution means for increasing the resolution of an image of an arbitrary subject imaged using images of a plurality of subjects imaged by the imaging element;
Image display means for displaying an image having a high resolution in the image high resolution means;
Adjusting means capable of manually adjusting the focusing means;
An imaging apparatus comprising: A focal point detection unit for detecting a focal point position of the imaging optical system using the image with high resolution by the image high resolution unit;
Mode setting means for switching between auto focus mode and manual focus mode;
Further comprising
The adjusting unit causes the focusing unit to adjust the focus of the imaging optical system based on the focus position detected by the focus detection unit when the autofocus mode is set by the mode setting unit. The imaging apparatus according to claim 2.   4. The image resolution increasing means is a super-resolution process for generating a resolution-enhanced image by using a plurality of images of the subject imaged by the imaging device. The imaging device described in 1.   The imaging apparatus according to claim 1, wherein the image resolution increasing unit includes a resolution increasing area selecting unit that selects an image area for performing an image resolution increasing process.   6. The image display device according to claim 2, wherein the image display means displays the high-resolution image and simultaneously enlarges and displays the image before the high-resolution image is increased in resolution. The imaging device according to any one of the above.   3. The display unit according to claim 2, wherein the display unit alternately displays the high-resolution image and an image obtained by enlarging the image before the high-resolution image is increased in resolution. The imaging device according to any one of 5.   6. The imaging apparatus according to claim 2, wherein the display unit displays the high-resolution images before and after adjustment by the adjustment unit side by side.   The display means displays the high-resolution images before and after adjustment by the adjustment means and the enlarged image of the high-resolution images before the high-resolution images are displayed side by side. An imaging apparatus according to any one of claims 2 to 5.   2. The image high-resolution means includes means for controlling the number of original images to be subjected to the high-resolution processing according to at least one of a photographing mode, a focal length, and an image magnification. The imaging apparatus in any one of thru | or 9. A focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
Using this high-resolution image, a procedure for detecting the in-focus position of the imaging optical system having the action of a lens,
A procedure for adjusting the focus of the imaging optical system based on the detected focal position,
Focus control program to execute. A focus control program for an imaging apparatus that electronically records an image of a subject,
On the computer,
Using a plurality of subject images captured by the image sensor, a procedure for increasing the resolution of any captured subject image;
The procedure to display this high resolution image,
A procedure for accepting an operation for adjusting the focus of the imaging optical system having the action of a lens;
A procedure for adjusting the focus of the imaging optical system in accordance with the operation;
Focus control program to execute.

Images