WO2008035635A1 - Dispositif d'imagerie et programme de commande de mise au point - Google Patents

Dispositif d'imagerie et programme de commande de mise au point Download PDF

Info

Publication number
WO2008035635A1
WO2008035635A1 PCT/JP2007/067957 JP2007067957W WO2008035635A1 WO 2008035635 A1 WO2008035635 A1 WO 2008035635A1 JP 2007067957 W JP2007067957 W JP 2007067957W WO 2008035635 A1 WO2008035635 A1 WO 2008035635A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
resolution
unit
focus
optical system
Prior art date
Application number
PCT/JP2007/067957
Other languages
English (en)
Japanese (ja)
Inventor
Takahiro Yano
Nobuyuki Watanabe
Eiji Furukawa
Shinichi Nakajima
Hideaki Furukawa
Original Assignee
Olympus Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Corporation filed Critical Olympus Corporation
Publication of WO2008035635A1 publication Critical patent/WO2008035635A1/fr

Links

Classifications

    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B13/00—Viewfinders; Focusing aids for cameras; Means for focusing for cameras; Autofocus systems for cameras
    • G03B13/32—Means for focusing
    • G03B13/34—Power focusing
    • G03B13/36—Autofocus systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28—Systems for automatic generation of focusing signals
    • G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • the present invention relates to an imaging apparatus that electronically records an image of a subject and a focus control program for such an imaging apparatus.
  • Japanese Patent Laid-Open No. 2003-179798 discloses an invention for confirming a subject in monitor display of a digital camera.
  • the release button when the release button is not pressed, the entire image of the subject imaged by the image sensor is displayed on the liquid crystal display device provided in the digital camera at the resolution of the liquid crystal device. If the release button is pressed halfway, the image power S in the area near the focus area center set in advance by the user is displayed in the enlarged area display with a higher resolution.
  • the edge sharpness of the image used for focusing is calculated.
  • a method is employed in which processing by a Laplacian filter is performed near the edge and the edge sharpness is calculated.
  • the present invention provides an imaging apparatus and a focus control program that can improve the focusing accuracy.
  • the present invention provides an imaging device that electronically records an image of a subject, an imaging element, an imaging optical system having a lens action, a focusing unit that focuses the imaging optical system, and the imaging An image resolution enhancement unit that uses a plurality of subject images captured by the element to increase the resolution of an arbitrary captured subject image, and an image that has been increased in resolution by the image resolution enhancement unit. Is used to adjust the focal point of the imaging optical system to the focusing unit based on the focal point position detected by the focal point detection unit.
  • an imaging device including an adjusting unit.
  • the present invention provides an imaging device that electronically records an image of a subject, an imaging element, an imaging optical system having a lens action, a focusing unit that focuses the imaging optical system,
  • An image pickup apparatus includes an image display unit that displays an image and an adjustment unit that can manually adjust the focusing unit.
  • the present invention is a focus control program for an imaging apparatus that electronically records an image of a subject, and is captured by a computer using a plurality of subject images captured by an imaging device.
  • a procedure for increasing the resolution of an image of an arbitrary subject a procedure for detecting the in-focus position of the imaging optical system having the function of a lens using the image with the increased resolution, and the detected in-focus position
  • a procedure for adjusting the focus of the imaging optical system and a focus control program for executing the procedure.
  • the present invention is a focus control program of an imaging apparatus that electronically records an image of a subject, and an arbitrary image captured using a plurality of subject images captured by an image sensor on a computer.
  • FIG. 1 is a schematic diagram of a digital camera as an imaging apparatus according to the first embodiment of the present invention. It is a typical block diagram.
  • FIG. 2A is a diagram showing an operation flowchart of the digital camera in the first embodiment.
  • FIG. 2B is a diagram showing a flowchart of a process for increasing the resolution of an image.
  • FIG. 3 is a diagram showing a flowchart of processing performed in the image displacement estimator.
  • FIG. 4 is a diagram for interpolating a discrete similarity map into a continuous similarity map. It is a figure which shows the example of the process using parabolic fitting.
  • FIG. 5 is a diagram showing a flowchart of the high resolution processing performed in the high resolution processing unit.
  • FIG. 6 is a schematic block diagram showing the configuration of a resolution enhancement processing unit.
  • FIG. 7 is a diagram showing an example of how the effect of high resolution due to the difference in sharpness appears in a high resolution image.
  • FIG. 8 is a conceptual diagram showing focusing by DFF processing.
  • FIG. 9 is a schematic block diagram of a digital camera as an imaging apparatus according to the second embodiment of the present invention.
  • FIG. 10 is a diagram showing an operation flowchart of the digital camera in the second embodiment.
  • FIG. 11A is a rear view of the digital camera for explaining the display method in the second embodiment.
  • FIG. 11B is a diagram showing a display example of the display method in the second embodiment.
  • FIG. 11C is a diagram showing a display example of the display method in the second embodiment.
  • FIG. 12A is a rear view of the digital camera for explaining another display method.
  • FIG. 12B is a diagram showing a display example of another display method.
  • FIG. 12C is a diagram showing a display example of another display method.
  • FIG. 13A is a diagram showing a display example of still another display method.
  • FIG. 13B is a diagram showing a display example of still another display method.
  • FIG. 13C is a diagram showing a display example of still another display method. BEST MODE FOR CARRYING OUT THE INVENTION
  • FIG. 1 is a block configuration diagram showing a configuration of a digital camera 10 as an imaging apparatus according to the first embodiment of the present invention.
  • This digital camera 10 has a lens system 12, a color filter array (CFA) 14, a CCD image sensor 16, an amplification circuit 18, an A / D conversion circuit 20, a buffer 22, a high resolution area selection unit 24, and a high resolution image.
  • the image resolution increasing unit 26 includes a use number control unit 26A, a reference image selecting unit 26B, an image displacement estimating unit 26C, a resolution increasing target image selecting unit 26D, and a resolution increasing processing unit 26E.
  • the thick and thin arrows shown in Fig. 1 indicate the video signal line and the control signal line, respectively.
  • an image photographed through the lens system 12, the CFA 14, and the CCD image sensor 16 is first amplified by the amplifier circuit 18. Subsequently, the video signal is input to the A / D conversion circuit 20 and converted into a digital signal. The output from the A / D conversion circuit 20 is input to the notch 22 via the video signal line.
  • a single plate CCD is assumed in which a Bayer-type primary color filter is disposed in front of the imaging system.
  • the Bayer-type primary color filter has 2 X 2 pixels as a basic unit, and a red (R) and blue (B) filter is arranged for each pixel, and a green (Gr, Gb) filter is arranged for two pixels. These green filters have the same power. In the present embodiment, these green filters are grouped as Gr and Gb for convenience.
  • the output from the node 22 is input to the image recording unit 36 and the high resolution area selection unit 24 via the video signal line.
  • the image recording unit 36 and the high resolution area selection unit 24 input to the image high resolution unit 26 via a video signal line.
  • the image recording unit 36 can further receive an output from the image resolution increasing unit 26. Outputs from the image resolution increasing unit 26 and the image recording unit 36 are input to the output unit 38 via a video signal line.
  • the external I / F unit 34 is bidirectionally connected to the control unit 32 via a control signal line.
  • the control unit 32 is connected to the CCD image pickup device 16, the amplifier circuit 18, and the A / D conversion circuit via a control signal line. Bidirectionally connected to the path 20, the in-focus detection unit 28, the motor 30, and the image enhancement unit 26. Therefore, the user can set shooting conditions such as ISO sensitivity via the external I / F unit 34.
  • the output from the image resolution increasing unit 26 is input to the in-focus detection unit 28 via the video signal line.
  • the focus detection unit 28 adjusts the lens system 12 to the focus position by controlling the motor 30 via the control line.
  • FIG. 2A is a diagram showing an operation flowchart of the digital camera 10.
  • step S10 After setting the shooting conditions, the user presses the release button halfway to start the pre-shooting mode (step S10).
  • the CCD image pickup device 16 takes a plurality of images.
  • the captured video signal is converted into a digital signal by the A / D conversion circuit 20 as described above. This digital signal is transferred to the high resolution area selection unit 24 via the buffer 22.
  • the resolution enhancement area selection unit 24 selects an image area on which the resolution enhancement process of the image is performed (step S12).
  • the means for selecting the region where the resolution of the image is to be increased is preferably a means for including a high-frequency component in the image for the purpose of selecting the region for adjusting the focal position of the image. For this reason, the edge region forces included in the image are mainly selected. In other words, a region having a lot of edge energy is preferentially selected as an edge region.
  • the edge energy is an integrated value after convolution of a differential filter such as a Laplacian filter.
  • the main subject to be focused is likely to be in the center of the captured image.
  • the subject is a moving object such as a person
  • the dynamic force of the subject can be detected by detecting the difference between the images of the multiple images. I can help.
  • the subject may be preferentially selected as the selection area.
  • the area selected is not limited to one place in the image. Instead, the region to be selected may be selected at a plurality of locations. Further, the above-described method for selecting a high resolution area is exemplary. Other high resolution area selection methods may be adopted.
  • the image high resolution unit 26 performs high resolution processing based on the selection area information selected as described above (step S14). Details of this high resolution processing will be described later.
  • the high resolution image output from the image high resolution unit 26 is input to the in-focus detection unit 28.
  • the in-focus detection unit 28 detects the in-focus using the high resolution image.
  • the in-focus detection unit 28 controls the motor 30 according to the detection result to adjust the focus of the lens system 12 (step S16). Details of this focus adjustment will be described later.
  • it is determined whether or not an in-focus state is obtained by this focus adjustment step S18. This determination method will also be described later. If it is not yet in focus, it is further confirmed whether or not the release button is half-pressed (step S20). If the release button force S is kept half pressed, the process returns to step S10 and the above process is repeated. If the release button is half-pressed, the operation ends.
  • step S 18 when it is determined in step S 18 that the in-focus state has been obtained, a standby state is maintained until the release button is fully pressed.
  • This standby state is a so-called focus lock. That is, it is determined whether or not the release button has been fully pressed (step S22). If the release button is not fully pressed, it is further determined whether or not the release button is half-pressed (step S24). If the half-press is not released, the process returns to step S22. Thus, the process loops through step S22 and step S24 until the release button is fully pressed. Processing is in a standby state. When it is determined in step S18 that the in-focus state has been obtained, it is desirable to notify the user that the in-focus state has been obtained by sound or display. If the release button is half-pressed in step S24, the operation is stopped.
  • step S26 the main photographing is performed with the focus adjusted as described above (step S26).
  • Real shooting The obtained image is transferred from the buffer 22 to the image recording unit 36.
  • This image is recorded by the image recording unit 36 (step S28).
  • the image recorded in the image recording unit 36 is transferred to the output unit 38 as necessary.
  • the digital camera 10 outputs an image.
  • the image recorded in the image recording unit 36 is transferred to the image high resolution unit 26 and used as a resource for image high resolution processing.
  • the resulting high-resolution image may be recorded in the image recording unit 36. Instead, after being transferred to the output unit 38, it may be output from the digital camera 10.
  • the high-resolution image obtained by the high-resolution processing in step S14 can be recorded in the image recording unit 36. This image can also be output by the output unit 38.
  • the image resolution increasing unit 26 includes the used number control unit 26A, the reference image selection unit,
  • the number-of-uses-control unit 26A uses the number of images used for high resolution processing according to at least one of the shooting mode, focal length, and image magnification specified by the external I / F unit 34. To control.
  • the number-of-use-number control unit 26A can control the number of sheets to be increased because the amount of light is small!
  • the use number control unit 26A can control to increase the number of images when the image magnification is low.
  • the use number control unit 26A can control to increase the number when the focal length is short. Prior to shooting, the used number control unit 26A selects one or more images to be used for higher resolution, including an image used as a reference for higher resolution.
  • FIG. 2B is a flowchart of the image resolution enhancement process. That is, one reference image serving as a reference for image displacement estimation is selected in the reference image selection unit 26B from a plurality of pre-captured images (trimmed) input to the image high resolution unit 26 (Ste S 141). Thereafter, an image corresponding position between the reference image selected by the reference image selection unit 26B and a pre-photographed image other than the reference image is obtained by the image displacement estimation unit 26C ( Step S 142). Subsequently, the resolution enhancement target image to be subjected to the resolution enhancement processing is selected (step S143). Finally, the higher resolution target image
  • FIG. 3 is a flowchart showing the processing of the image displacement estimation unit 26C that executes this image displacement estimation algorithm.
  • one reference image serving as a reference for image displacement estimation selected in step S141 is read (step S 1421).
  • an image sequence is generated by deforming the read reference image with a plurality of image displacements (step S 1422).
  • the standard image is 19 (27 out of 27) with a motion parameter of ⁇ 1 pixel in the horizontal and vertical directions and a motion parameter of ⁇ 1 ° [degree] in the rotation direction.
  • 8 types ((1 [pixel], 1 [pixel], -1 [degree]), (1 1 1), (1 1 1 1), (1 — 1 1), (— 1 1 1 —1), ( — 1 1 —1 1), (1 1 1), (— 1 —1 1)) can be omitted.
  • step S1423 one pre-captured image for which image displacement estimation with respect to the read standard image is performed is read as a reference image (step S1423).
  • a rough pixel position is associated between the read standard image and reference image by a pixel matching method such as region-based matching (step S 1424).
  • step S 1424 a pixel matching method
  • step S 1425 a similarity value between the image sequence generated in step S1422 and the reference image is calculated
  • step S1425 a discrete similarity map is created by using the relationship between the image displacement parameter and the similarity value calculated in step S1425 (step S1426).
  • Such similarity map power is schematically shown in FIG. In FIG. 4, the vertical axis represents the similarity. The smaller the similarity value, the higher the similarity.
  • the horizontal axis shows the motion parameters that deform the image.
  • an interpolated continuous similarity map is created by interpolating the discrete similarity map created in step S 1426.
  • the extreme values of this continuous similarity map are searched.
  • an extreme value (image displacement estimated value) of continuous similarity values is obtained (step S 1427).
  • step S 1428 it is confirmed whether image displacement estimation has been performed for all target reference images. If image displacement estimation is still performed and there is a reference image, the frame number of the next reference image is selected (step S 1429). In this case, the process returns to step S1423 and the above process is continued.
  • step S 1428 If image displacement estimation has been performed for all target reference images (step S 1428), the image displacement estimation processing is terminated.
  • step S143 the resolution enhancement target image is obtained by the resolution enhancement target image selection unit 26D in step S143. Selected. Subsequently, in step S144, the high resolution processing unit 26E uses the estimated image displacement values, the plurality of pre-photographed pre-photographed images, and the selection information of the high-resolution target image to increase the resolution. The resolution-enhanced target image is subjected to high-resolution processing.
  • FIG. 5 is a flowchart showing the high resolution processing performed in the high resolution processing unit 26E in step S144.
  • “super-resolution processing” is used as an algorithm for this high resolution processing. Details of this super-resolution processing
  • a plurality of low-resolution pre-photographed images are read for use in high-resolution image estimation (step S1441).
  • the pre-photographed image selected by the high-resolution target image selection unit 26D is set as a high-resolution target image.
  • An initial high-resolution image is created by performing interpolation processing such as linear interpolation and bicubic interpolation on this target image. (Step S 1442). This interpolation process can be omitted in some cases.
  • step S1 443 the pixel corresponding position between the target image and the other pre-captured image is clarified.
  • overlay processing is performed in a coordinate space based on the enlarged image of the target image.
  • a registration image y is generated. Details of the method for generating the registration image y are disclosed in the above-mentioned document “Speed-Up Algorithm for Reconstruction-type Super-Resolution Processing”.
  • each pixel value is converted into the enlarged coordinates of the target image.
  • the processing is performed on the grid point!
  • the image displacement between the target image and the other pre-captured images is the image displacement between the reference image and the target image estimated by the image displacement estimation unit 26C, and the image between the reference image and an image other than the target image. Generated by integrating inter-displacement.
  • PSF point spread function
  • OTF optical transfer function
  • CCD aperture a point spread function that takes into consideration imaging characteristics such as an optical transfer function (OTF) and a CCD aperture.
  • PSF is, for example, a force S that a Gauss function can be used easily.
  • evaluation function f (z) is minimized based on the pixel correspondence information in step S 1443 and the PSF information in step S 1444 (step S 1445).
  • This evaluation function f (z) is expressed by the following equation.
  • y is the registration image generated in step S 1443
  • z is the result image (high resolution image) of the target image with high resolution
  • A is the imaging system including PSF, CFA, etc.
  • g (z) is a constraint term in consideration of the smoothness of the image and the correlation between the colors of the image.
  • ⁇ is a weighting factor.
  • the steepest descent method is used to minimize the evaluation function.
  • a fine value of the evaluation function f (z) in the high-resolution image z is calculated.
  • the image is updated by adding the differential value to the high-resolution image z.
  • the minimum value of the evaluation function f (z) is obtained.
  • the fractional value of the evaluation function f (z) is as follows.
  • ⁇ represents the result image (high resolution image) of the first high resolution
  • step S 1445 it is determined whether or not the evaluation function f (z) obtained in step S 1445 has been minimized.
  • the evaluation function f (z) has not yet been minimized, the high-resolution image z is updated (step S 1447). Subsequently, the process
  • step S 1445 the evaluation function f (z) is minimized again.
  • step S 1445 if the evaluation function f (z) obtained in step S 1445 is minimized, it is considered that a high-resolution image z is obtained. In this case, the resolution increasing process is terminated.
  • FIG. 6 is a block diagram showing an example of the configuration of the high resolution processing unit 26E for performing the high resolution processing described above.
  • This high resolution processing unit 26E is composed of an interpolation enlargement unit 26E. El, image storage unit 26E2, PSF data holding unit 26E3, first convolution integration unit 26E4, registration image generation unit 26E5, image comparison unit 26E6, second convolution integration unit 26E7, regularization term calculation unit 26E8, updated image A generation unit 26E9 and a convergence determination unit 26E10 are provided.
  • the high-resolution target image selected by the high-resolution target image selection unit 26D is transferred to the interpolation enlargement unit 26E1.
  • the interpolation enlargement unit 26E1 performs interpolation enlargement of the target image (corresponding to step S1441). Examples of the interpolation expansion method used include bilinear interpolation and bicubic interpolation.
  • the interpolated and enlarged image is transferred to the image storage unit 26E2 and stored in the image storage unit 26E2.
  • the image stored in the image storage unit 26E2 is transferred to the first convolution integration unit 26E4.
  • this first convolution integration unit 26E4 convolution integration is performed between this image and the PSF data transferred from the PSF data holding unit 26E3 (this convolution integration is in the above formula (2)).
  • a T (y— corresponds to Az in the operation of Az)).
  • the target image is transferred to the registration image generation unit 26E5 together with a plurality of other pre-captured images.
  • the registration image generation unit 26E5 an overlay process is performed in a coordinate space based on the enlarged coordinates of the target image based on the image displacement information between the frames obtained by the image displacement estimation unit 26C. It is.
  • a registration image is generated (corresponding to step S1443 above).
  • each pixel value is placed on the closest lattice point of the enlarged coordinates of the target image. This is done by the process of placing pixels.
  • the image data subjected to the convolution operation by the first convolution integration unit 26E4 is transferred to the image comparison unit 26E6.
  • the same pixel is detected between the image data and the registration image generated by the registration image generation unit 26E5.
  • the difference value of the pixel value is calculated at the position.
  • difference image data is generated (corresponding to y ⁇ Az in the calculation of A T (y ⁇ Az) in the above equation (2)).
  • the difference image data generated by the image comparison unit 26E6 is transferred to the second convolution integration unit 26E7.
  • convolution integration is performed between the difference image data and the PSF data given by the PSF data holding unit 26E3 (A T (y ⁇ Az) in the above equation (2)) (Corresponds to y z ⁇ (y—Az) in).
  • a regularized image is generated by the regularization term calculation unit 26E8 from the image stored in the image storage unit 26E2. That is, in the regularization term calculation unit 26E8, for example, the RGB power is also subjected to color conversion processing to YCrCb on the image stored in the image storage unit 26E2.
  • the Cr and Cb components are subjected to convolution calculation processing of a frequency wide-pass filter (Laplacian filter) to generate a regularized image.
  • the regularized image generated by this method in the updated image generation unit 26E9 the foreseeing information of the image that “the color difference component of the image is generally a smooth change” is used. Therefore, a high-resolution image with suppressed color difference can be obtained stably (corresponding to partial differentiation by z of g (z) in the above equation (2)).
  • the image generated by the second convolution integrator 26E7, the image stored in the image storage unit 26E2, and the image generated by the regularization term calculation unit 26E8 are updated image generation units. Forwarded to 26E9.
  • the updated image generation unit 26E9 generates an updated image by generating a weighted image of these three images (corresponding to the above equation (3)).
  • the generated updated image is transferred to the convergence determination unit 26E10.
  • the convergence determination unit 26E10 performs convergence determination. In this convergence determination, it can be determined that the image update operation has converged when the number of iterations required for convergence exceeds a certain number.
  • the convergence determination unit 26E10 determines that the update operation has converged, the updated image is output to the outside as a high-resolution image.
  • FIG. 7 is a diagram showing a comparison between the final high resolution image obtained in this way and the area selection image (low resolution image) selected by the high resolution area selection unit 24. is there.
  • the right side is a low-resolution image before high resolution.
  • the left side shows a high-resolution image that is high-resolution based on the low-resolution image.
  • the upper side is an image with high sharpness, while the lower side is an image with low sharpness.
  • the resolution restoration effect is small for an image with low sharpness as a result of the high-resolution processing.
  • High-quality images have the property of high resolution restoration effect.
  • the high-resolution image with high sharpness has a greater effect on the corresponding low-resolution image than the high-resolution image with low sharpness. .
  • the nature of the algorithm itself favors the control for focusing described below.
  • the resolution enhancement processing in the image resolution enhancement unit 26 the resolution of the selected areas of the plurality of images selected by the resolution enhancement area selection unit 24 is increased.
  • the high resolution image obtained by the image high resolution unit 26 is transferred to the in-focus detection unit 28.
  • step S16 the focus detection unit 28 performs a DFF (Depth From Focus) process as shown in FIG. 8 to control the motor 30 to adjust the focus position.
  • the contrast method is a typical DFF process.
  • the edge sharpness of the high-resolution image is detected by convolving a high-pass filter such as a Laplacian filter with the image.
  • the in-focus detection unit 28 controls the motor 30 in the direction of increasing the sharpness so as to align the focus.
  • the sensitivity of the edge sharpness is improved by performing the high resolution processing. As a result, the accuracy of focusing is improved.
  • the pre-photographed image is transferred to the high resolution area selection unit 24 via the amplifier circuit 18, the A / D conversion circuit 20, and the buffer 22.
  • the pre-photographed image is transferred to the image resolution increasing unit 26 after the area is selected. The resolution of the selected area is increased.
  • the resulting high-resolution image is transferred to the in-focus detection unit 28.
  • the in-focus detection unit 28 detects the sharpness of the high-resolution image. Subsequently, the in-focus detection unit 28 controls the motor 30 to adjust the focus.
  • the focus point detection unit 28 determines that the focal position of the lens system 12 is in focus when the sharpness reaches the maximum value.
  • the digital camera 10 enters the main photographing mode by fully pressing the user 1S release button, and the subject's book is recorded. Take a picture.
  • the actually captured image is recorded in the image recording unit 36 from the buffer 22 via the video signal line.
  • the recorded image is transferred to the image high resolution unit 26 via the video signal line as necessary. This transferred image is used as a resource for image resolution enhancement processing.
  • the transferred image is transferred to the output unit 38 via the video signal line, and as a result, the image output from the digital camera 10 is performed.
  • the focus adjustment using the high resolution processing when the high resolution algorithm (super-resolution processing algorithm) shown in FIG. 5 is employed has been described above.
  • the high resolution algorithm is not limited to this super-resolution processing algorithm, and other algorithms may be adopted.
  • FIG. 9 is a block diagram schematically showing the configuration of the digital camera 10 as the imaging apparatus according to the second embodiment of the present invention.
  • the digital camera 10 in the present embodiment has the same configuration as that in the first embodiment.
  • the same reference numerals as those shown in FIG. 1 are attached to the same components as those shown in FIG. is there.
  • the digital camera 10 according to the present embodiment includes an image display unit 40 such as an LCD which is connected to the buffer 22, the high resolution area selection unit 24, and the image high resolution unit 26 and displays video signals output from them. I have.
  • the external I / F unit 34 is provided with a focusing mechanism 34A. Similar to FIG. 1, the thick and thin arrows shown in FIG. 9 indicate the video signal line and the control signal line, respectively.
  • FIG. 10 is a view showing an operation flowchart of the digital camera 10 in the second embodiment.
  • in-focus detection is performed by selecting either autofocus or manual focus.
  • Step S30 When the user presses the release button halfway after setting the shooting conditions, it is first determined whether or not the manual focus mode is selected! / (Step S30). If it is determined that the manual focus mode is selected! /, NA! /, That is, the autofocus mode is selected, the autofocus process is executed (step S32). In this autofocus process, the processes from step S10 to step S28 in the first embodiment are applied as they are. Therefore, the description is omitted.
  • step S34 the pre-photographing mode is started (step S34).
  • a plurality of pre-photographed images are photographed in the same manner as in step S10. These pre-photographed images are transferred to the high resolution area selection unit 24 via the buffer 22.
  • the high resolution area selection unit 24 selects an area of the image to be subjected to the high resolution process of the image (step S36). Since this high resolution area selection process is the same process as in step S12, detailed description thereof is omitted.
  • step S40 The high resolution image output from the image high resolution unit 26 is input to the image display unit 40.
  • the image display unit 40 displays a high resolution image (step S40).
  • the user determines whether or not the image is in focus by viewing the high-resolution image displayed on the image display unit 40.
  • the user can perform full shooting by pressing the release button fully. That is, it is determined whether or not the release button has been fully pressed after the high resolution image is displayed in step S40 (step S42). If it is determined that the release button has been pressed all the way, the main shooting mode starts. Subsequently, actual photographing is performed (step S44).
  • the captured image data is recorded in the image recording unit 36 (step S46). If necessary, the image recorded in the image recording unit 36 is transferred to the output unit 38 and the digital camera 10 outputs the image.
  • the image recorded in the image recording unit 36 is transferred to the image high resolution unit 26 and used as a resource for image high resolution processing.
  • the resulting high-resolution image is recorded in the image recording unit 36.
  • the high resolution image is transferred to the output unit 38 and output from the digital camera 10.
  • the high-resolution image can be displayed on the image display unit 40.
  • the user controls the motor 30 via the control unit 32 and adjusts the focus of the lens system 12 by operating the focusing mechanism 34A provided in the external I / F unit 34. .
  • step S48 it is determined whether or not the user has operated the focusing mechanism 34A.
  • step S50 it is further determined whether or not the release button is half-pressed.
  • step S48 if it is determined that the user has operated the focusing mechanism 34A, the motor 30 is controlled via the control unit 32 according to the operation, and the lens system 12 The focus is adjusted (step S52). Thereafter, it is determined whether or not the user continues to press the release button halfway (step S54). If the release button has been pressed halfway, the process returns to step S34. As a result, the operations for manual focus adjustment in steps S34 to S52 are repeated with the focus adjusted by the user. On the other hand, if it is determined in step S54 that the half-press of the release button has been completed, shooting is terminated.
  • the external I / F unit 34 of the digital camera 10 in the second embodiment is configured so that the user can select either manual focus or autofocus shooting mode.
  • the selection of the photographing mode can be performed by a switching unit such as a switching switch provided in the external I / F unit 34.
  • the user sets this switching unit in advance before entering the above flow for focusing by pressing the release button halfway.
  • the user sets the switching unit to manual focus or auto force, and then presses the release button halfway.
  • the user can switch between manual focus and auto focus during the in-focus flow by switching this switching section while pressing the release button halfway. is there
  • the display of the high resolution image on the image display unit 40 is performed as shown in FIG. 11A, for example.
  • FIG. 11A an image of the digital camera body 10A in the second embodiment is displayed.
  • the image display unit 40 is configured as a liquid crystal display panel 4OA provided on the back surface of the digital camera body 10A.
  • a part of the image display area 40B of the liquid crystal display panel 40A is used as a high resolution image display area 40C.
  • the high resolution image is displayed in the high resolution image display area 40C.
  • information on parameters such as an arbitrary image, shot speed, ISO sensitivity, focal length, etc. can be displayed.
  • the captured image at the current time transferred to the notifier 22 may be displayed.
  • FIG. 11A includes a power switch 34B, a release button 34C, and an operation button 34D as a part of the digital camera body 10A force external I / F unit 34! /, A force S-- is shown as an example. /!
  • FIGS. 11B and 11C show display examples of high-resolution images on the image display unit 40.
  • FIG. As shown in these figures, a part of the image display area 40B is a high-resolution image display area 40C.
  • a super-resolution algorithm is used as an algorithm for high resolution processing, sharpness is high! /, And when it is low! /, There is a large difference in sharpness. As a result, it is possible to easily confirm whether or not the focus position is correct.
  • the user when the sharpness of the displayed high-resolution image is low as shown in FIG. 11C, the user operates the focusing mechanism 34A to increase the resolution as shown in FIG. 11B. Adjust the focus of the digital camera 10 so that the sharpness of the image increases.
  • the high-resolution image displayed in the high-resolution image display area 40C becomes an image with high sharpness as illustrated in FIG. Judge that it has burnt and press release button 34C fully.
  • the digital camera 10 enters the main shooting mode and can perform the main shooting of the subject. By such an operation, the user can shoot an image with an in-focus position. As described with reference to FIG.
  • a resolution restoration effect is obtained for an image with low sharpness as a result of the high resolution processing.
  • it has the property of high resolution restoration effect.
  • the advantageous effect of improving the sensitivity of the sharpness of the wedge appears, and the sensitivity of the focus adjustment is improved.
  • FIG. 12A is a diagram showing another display method, in which the high-resolution image display area 40C and the low-resolution image display area 40D are arranged in the image display area 40B of the liquid crystal display panel 40A, thereby increasing the resolution. And the low-resolution area selection image selected by the high-resolution area selection unit 24 are displayed side by side. In this case, the area selection image is enlarged and displayed to the same size as the high resolution image by the bilinear method or the like in order to compare the image with the high resolution image.
  • the reference numerals shown in FIG. 11A that are identical to the reference signs shown in FIG. 12A indicate the same components as in FIG. 11A! /.
  • an image used for image display is a high-resolution image generated from a plurality of pre-photographed images and an enlarged region selection image. Therefore, it is clear that the sharpness is improved in the high resolution image.
  • an algorithm for super-resolution processing is used as an algorithm for high resolution processing, it is possible to compare two images as compared to the case where only an enlarged image of one image is used for image display as in the past. The difference in sharpness between high and low sharpness is clear. As a result, by comparing the high resolution image and the low resolution image, it is possible to easily confirm whether or not the focus position is correct.
  • FIGS. 12B and 12C show display examples on the image display unit 40.
  • the high resolution image shown in FIG. 12B has high sharpness, while the high resolution image shown in FIG. 12C has low sharpness.
  • the focusing mechanism 34A is used, Adjust the focus.
  • a low-resolution area selection image is displayed in the low-resolution image display area 40D. Therefore, when comparing the above-mentioned image resolution, if the sharpness is high, the drop in resolution is large. If the sharpness is low, the drop in resolution is small! /, And! / The characteristics of super-resolution processing are utilized. As a result, it is possible to make it easier to check whether the focal position is in focus. As a result, when the high-resolution image displayed in the high-resolution image display area 40C becomes an image with high sharpness as illustrated in FIG. 12B, the user is in focus. Judge. In this case, when the release button 34C is fully pressed, the user performs the main shooting of the subject with the digital camera 10 as the main shooting mode. By such an operation, the user can take an image with a focal position.
  • a display method as shown in FIG. 13A, B or C may be adopted.
  • two high-resolution image display areas 40C1 and 40C2 are arranged in the image display area 40B.
  • the first high-resolution image display area 40C1 a high-resolution image generated from a plurality of images taken by the previous focus adjustment is displayed.
  • the second high-resolution image display area 40C2 a high-resolution image with the focus adjusted once before the high-resolution image displayed in the first high-resolution image display area 40C1 is displayed. Is done.
  • FIG. 11A only one high-resolution image generated from a plurality of images taken by the previous force adjustment was displayed.
  • the resolution-enhanced images with the focus adjusted once before are displayed side by side so that the user can more easily determine the focus.
  • the super-resolution processing algorithm as a result of the high resolution processing, an image with a low sharpness is obtained for an image with a high sharpness with a little effect of restoration of resolution. It has the property that the feeling restoration effect is high. Therefore, by displaying side by side as in the present display method, the resolution restoration effect of the two images can be compared more easily.
  • the explanation display 40E (before adjustment) and "after adjustment” is provided so that it can be seen whether the resolution enhancement image of V or deviation is the resolution enhancement image generated by the previous focus adjustment.
  • the caption is displayed in the vicinity of the first and second high resolution image display areas.
  • the display position of this explanation display 40E and whether or not to display this explanation display 40E may be selectable by the user via the external I / F section 34. Further, the explanation display 40E is not limited to “before adjustment” and “after adjustment” as shown in the figure, but may be other things. Further, the display positions of the first high-resolution image display area 40C1 and the second high-resolution image display area 40C2 may be adjustable by the user via the external I / F unit 34. . Note that any image, shot speed, ISO sensitivity, focal length, etc.
  • the parameter information related to the image can be displayed, and in particular, the captured image at the current time transferred to the buffer 22 may be displayed.
  • the high-resolution image by the high-resolution image display area 40C and the low-resolution area selection image by the low-resolution image display area 40D are enlarged. Display is displayed alternately. In the display method shown in FIG. 12A, these two display areas are displayed side by side. In contrast, in this display method, when the information is displayed in an area outside the display area by displaying alternately, a captured image at the current time transferred to the nother 22 is displayed. In this case, there is an advantage that information outside these display areas can be easily confirmed.
  • the resolution restoration effect is reduced for images with low sharpness and the sharpness restoration effect is low for images with low sharpness. Is expensive.
  • a high-resolution image generated using a plurality of images displayed in the high-resolution image display area 40C and a low-resolution image composed of one image displayed in the low-resolution image display area 40D The image resolution is compared. In this case, if the sharpness is high, the drop in resolution is large, and if the sharpness is low, the drop in resolution is small! / It is possible to confirm whether or not this display method displays two images alternately.
  • the user can adjust the display times of the high-resolution image display area 40C and the low-resolution image display area 40D via the external I / F unit 34, respectively. Furthermore, the user can set the image display areas of the high-resolution image display area 40C and the low-resolution image display area 40D to different locations via the external I / F unit 34. In addition, the time interval is set and the time interval is adjusted while neither of the image display regions is displayed during the display change of the high resolution image display region 40C and the low resolution image display region 40D. I can do it.
  • the captured image at the current time transferred to the buffer 22 as described above may be displayed.
  • two high-resolution image display areas 40C1 and 40C2 and two low-resolution image display areas 40D1 and 40D2 are arranged in the image display area 40B.
  • the first high-resolution image display area 40C1 a high-resolution image generated from a plurality of images taken by the previous focus adjustment is displayed, and the second high-resolution image display area 40C2 is displayed.
  • the high resolution image with the focus adjusted once before the high resolution image displayed in the first high resolution image display area 40C1 is displayed.
  • the low-resolution area selection image captured with the focus of the high-resolution image displayed in the first high-resolution image display area 40C1 is displayed.
  • a low-resolution area selection image captured with the focus of the high-resolution image displayed in the second high-resolution image display area 40C2 is displayed.
  • the display areas are arranged so that the high-resolution image and the area selection image generated with the same focus are displayed side by side.
  • explanation display 40E of “after adjustment” and “before adjustment” is performed, respectively.
  • an image with a low sharpness has little effect on restoration of an image with a low sharpness
  • an image with a high resolution has a low resolution. It has the property of high restoration effect.
  • the effect of restoring the resolution of the two images can be more easily compared by displaying them side by side as in the present display method.
  • High resolution image display area Low resolution image consisting of high resolution image generated using multiple images displayed in 40C1 and 40C2, and one image displayed in low resolution image display area 40D1 and 40D2. The resolution of the images is compared with each other.
  • the explanation display 40E may be another display that does not necessarily need to be “before adjustment” and “after adjustment”.
  • the user can cancel these displays via the external I / F unit 34.
  • parameters related to shooting such as an arbitrary image, shot speed, ISO sensitivity, focal length, etc.
  • Information can be displayed, in particular, the current time taken image transferred to the buffer 22 as described above may be displayed! /.
  • the display method described with reference to FIG. 11A, FIG. 12A, FIG. 13A, B, and C may be configured to be selectable by the user via the external I / F unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optics & Photonics (AREA)
  • Studio Devices (AREA)
  • Focusing (AREA)
  • Automatic Focus Adjustment (AREA)
  • Image Processing (AREA)

Abstract

La présente invention concerne un appareil de prise de vues numérique sous la forme d'un dispositif d'imagerie. L'appareil de prise de vues numérique est équipé d'un élément de formation d'images à dispositif à couplage des charges ; un système de lentille ; un moteur pour la mise au point du système de lentille; une section pour la sélection d'une zone destinée à être en haute résolution et une section pour former une image en haute résolution , les deux permettant la formation d'une image d'un objet séparé mis en image en haute résolution grâce à l'utilisation d'une pluralité d'images d'objets captées par l'élément de formation d'images à dispositif de couplage des charges ; et une section de détection de point focal, qui détecte une position de point focal du système de lentille au moyen de l'image destinée à être en haute résolution et pour ajuster le point focal du système de lentille par le moteur, en fonction du point focal détecté.
PCT/JP2007/067957 2006-09-22 2007-09-14 Dispositif d'imagerie et programme de commande de mise au point WO2008035635A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2006257620A JP2008079123A (ja) 2006-09-22 2006-09-22 撮像装置及びフォーカス制御プログラム
JP2006-257620 2006-09-22

Publications (1)

Publication Number Publication Date
WO2008035635A1 true WO2008035635A1 (fr) 2008-03-27

Family

ID=39200461

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2007/067957 WO2008035635A1 (fr) 2006-09-22 2007-09-14 Dispositif d'imagerie et programme de commande de mise au point

Country Status (2)

Country Link
JP (1) JP2008079123A (fr)
WO (1) WO2008035635A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104641625A (zh) * 2012-09-19 2015-05-20 富士胶片株式会社 图像处理装置、摄像装置、图像处理方法及图像处理程序
CN113163114A (zh) * 2021-03-29 2021-07-23 无锡闻泰信息技术有限公司 一种图像对焦方法、装置、设备及介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5412849B2 (ja) * 2009-01-28 2014-02-12 株式会社ニコン オートフォーカス装置
JP5627256B2 (ja) * 2010-03-16 2014-11-19 キヤノン株式会社 画像処理装置、撮像装置および画像処理プログラム
TWI470300B (zh) * 2012-10-09 2015-01-21 Univ Nat Cheng Kung 影像對焦方法與自動對焦顯微裝置

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0965194A (ja) * 1995-08-18 1997-03-07 Matsushita Electric Ind Co Ltd 撮影装置
JPH10173966A (ja) * 1996-12-13 1998-06-26 Canon Inc 撮像装置
JP2002094869A (ja) * 2000-09-14 2002-03-29 Olympus Optical Co Ltd 電子カメラ
JP2002112095A (ja) * 2000-09-29 2002-04-12 Minolta Co Ltd デジタルスチルカメラ
JP2002214524A (ja) * 2001-01-23 2002-07-31 Minolta Co Ltd デジタル撮像装置および記録媒体
JP2003101867A (ja) * 2001-09-19 2003-04-04 Olympus Optical Co Ltd 撮像装置
JP2004186933A (ja) * 2002-12-03 2004-07-02 Ricoh Co Ltd デジタルカメラ
JP2006030972A (ja) * 2004-06-18 2006-02-02 Canon Inc 撮像装置及び撮像方法
JP2006178173A (ja) * 2004-12-22 2006-07-06 Matsushita Electric Ind Co Ltd カメラ
JP2006189572A (ja) * 2005-01-05 2006-07-20 Canon Inc オートフォーカス装置

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0965194A (ja) * 1995-08-18 1997-03-07 Matsushita Electric Ind Co Ltd 撮影装置
JPH10173966A (ja) * 1996-12-13 1998-06-26 Canon Inc 撮像装置
JP2002094869A (ja) * 2000-09-14 2002-03-29 Olympus Optical Co Ltd 電子カメラ
JP2002112095A (ja) * 2000-09-29 2002-04-12 Minolta Co Ltd デジタルスチルカメラ
JP2002214524A (ja) * 2001-01-23 2002-07-31 Minolta Co Ltd デジタル撮像装置および記録媒体
JP2003101867A (ja) * 2001-09-19 2003-04-04 Olympus Optical Co Ltd 撮像装置
JP2004186933A (ja) * 2002-12-03 2004-07-02 Ricoh Co Ltd デジタルカメラ
JP2006030972A (ja) * 2004-06-18 2006-02-02 Canon Inc 撮像装置及び撮像方法
JP2006178173A (ja) * 2004-12-22 2006-07-06 Matsushita Electric Ind Co Ltd カメラ
JP2006189572A (ja) * 2005-01-05 2006-07-20 Canon Inc オートフォーカス装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104641625A (zh) * 2012-09-19 2015-05-20 富士胶片株式会社 图像处理装置、摄像装置、图像处理方法及图像处理程序
CN104641625B (zh) * 2012-09-19 2018-05-11 富士胶片株式会社 图像处理装置、摄像装置及图像处理方法
CN113163114A (zh) * 2021-03-29 2021-07-23 无锡闻泰信息技术有限公司 一种图像对焦方法、装置、设备及介质

Also Published As

Publication number Publication date
JP2008079123A (ja) 2008-04-03

Similar Documents

Publication Publication Date Title
JP5544047B2 (ja) 画像処理装置、方法及びプログラム並びに立体撮像装置、携帯電子機器、プリンタ及び立体画像再生装置
JP5893713B1 (ja) 撮像装置、撮像方法、処理プログラム
JP4524717B2 (ja) 画像処理装置、撮像装置、画像処理方法及びプログラム
JP4665718B2 (ja) 撮影装置
JP2008306651A (ja) 撮像システムおよびプログラム
WO2008041522A1 (fr) Dispositif de traitement d'image, programme de traitement d'image, procédé de production d'image et support d'enregistrement
JP6825073B2 (ja) 撮像装置、制御方法、およびプログラム
KR20100039430A (ko) 화상 처리기, 화상 처리 방법, 디지털 카메라 및 촬영 장치
JP3395770B2 (ja) デジタルスチルカメラ
EP3383023B1 (fr) Appareil d'imagerie, procédé de commande et support d'informations non transitoire
JP2008077501A (ja) 画像処理装置及び画像処理制御プログラム
JP3678160B2 (ja) 画像処理装置、表示装置、プログラムおよび記憶媒体
JP2009111596A (ja) 撮像装置
JP5212046B2 (ja) デジタルカメラおよび画像処理装置並びに画像処理プログラム
WO2008035635A1 (fr) Dispositif d'imagerie et programme de commande de mise au point
WO2005085927A1 (fr) Appareil et procede de commande de mise au point automatique, support d'enregistrement et programme
JP2018037857A (ja) 画像処理装置、画像処理方法およびプログラム
JP4586707B2 (ja) 画像処理装置、電子カメラおよび画像処理プログラム
JP3444281B2 (ja) 画像処理システム、撮像装置、画像処理方法及びその方法の処理プログラムが記録された記録媒体
JP5100606B2 (ja) 撮像装置
JP5117889B2 (ja) 画像処理装置及び画像処理方法
JP2008109482A (ja) 撮像装置、画像記録処理方法及び画像記録処理プログラム
JP2015233211A (ja) 撮像装置およびその制御方法ならびにプログラム
JP6961423B2 (ja) 画像処理装置、撮像装置、画像処理装置の制御方法、プログラムおよび記録媒体
JP2020160773A (ja) 画像処理装置、撮像装置、画像処理方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07807361

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07807361

Country of ref document: EP

Kind code of ref document: A1