JP2011124712A - Image processing device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To implement contour enhancement in consideration of a subject distance for the contour enhancement correction of image data captured by an imaging device. <P>SOLUTION: A CPU 40 calculates a distance difference between a focused subject that is a subject within a predetermined focusing area and a peripheral subject that is a subject within each ambient area, and stores a distance difference corresponding to each ambient area in an SDRAM 48. Then, the CPU 40 determines whether or not the distance difference corresponding to each ambient area is below a predetermined threshold, and if not, on the basis of second conversion property information (having a conversion property determined in such a manner that a slope of a value change of output data versus value change of input data is reduced as a value of the distance difference from the focused subject corresponding to the image data of each ambient area becomes large), the CPU controls a digital signal processing section 24 to convert the intensity of an contour component indicated by contour component data of an ambient area with the distance difference equal to or higher than a predetermined threshold for YC data after primary image imaging to make contour enhancement. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

The present invention relates to edge enhancement correction of image data acquired by an imaging apparatus.

  In Patent Document 1, the image processing unit changes the tone correction characteristics of the main subject area from other areas. For example, the image processing unit executes gradation correction by applying a gradation characteristic table having a gradation curve different from that of the other areas to the main subject area of the captured image.

  In Patent Document 2, the image quality improvement apparatus discriminates the waveform of the input signal. When the input signal is a sine wave, cosine wave, modulation wave or the like having a constant amplitude, the edge enhancement component is not added and the input signal is pulsed. In addition, in the case of a signal including a rising portion of a waveform such as a bar pulse, and a sine wave, cosine wave, or modulated wave whose amplitude changes, an edge emphasis component is added to the slope portion of the waveform. Therefore, this image quality improvement device can add an edge enhancement component only to a portion that needs edge enhancement, and enhances the sharpness of the image quality by enhancing the contour portion of the reproduced image without visually unnaturalness. Thus, the image quality can be improved.

  In Patent Document 3, a minute convex lens array is provided in a matrix in front of an imaging surface, and a subject image is incident on two different imaging elements, and a focus state is detected according to a phase difference between output signals of the two imaging elements. It is an example of the technique to do.

  In Patent Documents 4 and 5, when the contrast information of the subject is input to the system controller, the determination / arithmetic processing unit recognizes the in-focus state of the imaging device from the information, and the aperture compensation circuit of the process processing unit The image is formed in accordance with the in-focus state by changing the frequency characteristic of.

  In Patent Document 6, an exit pupil of an imaging lens is divided into two by a spectacle lens, and a light beam that has passed through the two divided regions is imaged on a storage type photoelectric conversion element array (line sensor). This is an example of a technique for specifying an in-focus state by A / D converting an output signal of a sensor with an A / D converter and calculating a displacement of a relative position of those images by a computer. Specifically, the left and right images secondarily formed by the spectacle lens are received by different line sensor groups. The line sensor pixel output is output through various processing circuits. After the line sensor pixel output from the light receiving sensor is converted into a digital signal by the A / D converter, the microcomputer performs a correlation calculation between the A image and the B image, and the defocused state (or distance information) of the subject. Is detected. Further, Patent Document 6 discloses that a multipoint focus detection apparatus capable of independently detecting a focus state with respect to a plurality of areas on an imaging screen as a focus detection apparatus is configured to measure each point in order to perform multipoint distance measurement. It is assumed that an independent line sensor is provided in the distance area, and that an AGC area and a distance measurement calculation area are set for each line sensor.

  Patent Document 7 discloses a focus control based on a TTL (Through The Lens) phase difference detection method, that is, a position where a defocus amount is calculated from an image interval between two images and driven by the defocus amount calculated from the current lens position. This is an example of a technique for stopping the lens.

  Patent Document 8 shows an example of a technique for dividing an imaging screen, detecting the focus of a subject for each divided region, and detecting the image shift amount. Since the image shift amount depends on the distance to the subject, the image shift amount can be technically equated with the subject distance.

  Patent document 9 shows an example of an outline emphasis process. Patent Document 10 shows an example of the relationship between the lens performance with respect to the focal length, particularly the relationship between the optical performance in the central region and the peripheral region of the shooting angle of view according to the focal length. Further, Patent Document 10 changes the gain value and coring level of the contour correction process so as to compensate for the image performance degradation when used outside the performance guarantee range of the photographing optical system. Patent Document 11 shows an example of a face detection technique.

JP 2007-282119 A, paragraph 0062 Japanese Patent No. 2530249 (Japanese Patent Laid-Open No. 4-139966) Japanese Patent No. 2959142 Japanese Patent No. 3469590 (Japanese Patent Laid-Open No. 5-183796) JP 2002-287846 A (JP 2004-62188 A) JP 2006-285639 A JP 2006-276699 A JP-A-4-73731 JP 2006-129176 A JP 2002-64745 A, paragraph 0109, paragraph 0116, FIG. Japanese Patent Publication No. 2009-104427

  In the prior art, contour enhancement is uniformly applied to the entire screen, the detected face, the main subject, and the like. Thus, for example, in the prior art, contour enhancement is performed on the background even when the main subject is in focus and the background is not in focus. As a result, the outline of the blurred part is emphasized, and an undesirable image is obtained. The present invention performs contour enhancement in consideration of subject distance.

  The present invention relates to an input unit for inputting image data obtained by photoelectrically converting a subject imaged via an imaging lens by an imaging element, a distance calculation unit for calculating a distance to the subject, and a distance calculation unit There is provided an image processing apparatus comprising: a contour emphasis processing unit that performs contour emphasis processing with an intensity corresponding to the calculated distance for image data input by an input unit.

  Preferably, the image processing apparatus includes a focusing control unit that controls a position of the imaging lens so that a subject existing in a predetermined focusing area of the image data is focused, and the distance calculation unit includes a predetermined focusing of the image data. The contour enhancement processing unit calculates the difference between the distance to the focused subject that is the subject in the focal area and the distance to the peripheral subject that is a subject other than the focused subject. Intensity contour enhancement processing according to the difference from the distance to the subject is performed on the peripheral subject.

  Preferably, the contour emphasizing processing unit performs contour emphasizing processing with decreasing intensity as the difference between the distance to the focused subject and the distance to the surrounding subject increases.

  That is, according to the present invention, a natural image can be obtained by weakening the contour enhancement according to the distance difference.

  Preferably, the image processing apparatus includes a mode setting unit that sets a shooting mode, and the contour emphasis processing unit determines whether the distance to the in-focus subject and the distance to the surrounding subject are in accordance with the shooting mode set by the mode setting unit. An edge enhancement process with an intensity corresponding to the difference is performed on the peripheral subject.

  Preferably, the contour emphasis processing unit performs the contour emphasis processing with weaker intensity than the focused subject on the peripheral subject in accordance with the shooting mode set by the mode setting unit being the person mode.

  That is, in the case of the person mode, it is possible to weaken the contour emphasis of the subject close to the person, thereby relatively enhancing the person and obtaining an image suitable for the person mode.

  Preferably, the contour emphasis processing unit performs the contour emphasis processing on the peripheral subject with the same intensity as that of the focused subject in accordance with the shooting mode set by the mode setting unit being the landscape mode.

  That is, in the landscape mode, the image quality of the distant view can be improved and the image suitable for the landscape mode can be obtained by not weakening the outline enhancement of the distant view even when focusing on a short distance.

  Preferably, the image processing apparatus includes an in-focus area setting unit that sets a predetermined in-focus area in the image data, and an image between the predetermined in-focus area set by the in-focus area setting unit and the central portion of the image data. A determination unit that determines a difference in position on the data, and the contour enhancement processing unit performs an edge enhancement process that is weaker than the region other than the central portion of the image data according to the difference determined by the determination unit. Perform in the center of the data.

  That is, since the lens performance is good at the center of the screen, it is possible to balance the entire image by weakening the edge enhancement at the center of the image data.

  The contour emphasis processing unit performs a contour emphasis process with an intensity corresponding to the focal length of the zoom lens at the center of the image data.

  In other words, taking into consideration that the lens performance at the center of the screen changes according to the focal length of the zoom lens, the intensity of contour emphasis at the center of the screen can be changed according to the focal length of the zoom to balance the entire image.

  The distance calculation unit calculates the distance to the subject based on the defocus amount of the region that substantially covers the entire image data, and the focus control unit captures an image based on the defocus amount of the predetermined focus region Control the position of the lens.

  Here, the region that substantially covers the entire image data does not mean that the entire image data is completely covered without a gap, but includes covering the image data densely or roughly.

  The present invention includes a step in which an image processing apparatus inputs image data obtained by photoelectrically converting a subject imaged through an imaging lens by an imaging device, a step of calculating a distance to the subject, and a calculation An image processing method is provided for executing a contour enhancement process with an intensity according to the distance performed on input image data.

  The present invention provides an image processing program for causing an image processing apparatus to execute this image processing method.

  According to the present invention, the contour emphasis processing with the intensity corresponding to the distance to the subject is performed. Therefore, for example, it is possible to weaken the contour enhancement processing for a subject having a large distance difference compared to the focused subject, that is, a subject that is greatly out of focus, and obtain a natural image.

The block diagram which shows embodiment of an imaging device (digital camera) A diagram showing an example of the configuration of a CCD (example of Bayer array) A diagram showing a configuration example of a CCD (another example of a Bayer array) Diagram showing CCD configuration example (honeycomb array example) Enlarged view of the main part of CCD The figure which showed the cross section of CCD typically The figure which shows the relationship between a focus state and the output (phase difference) of the pixel A and the pixel B for phase difference detection The figure which shows the structural example of other CCD Flowchart of photographing process according to the first embodiment Flowchart of photographing process of the second embodiment Flowchart of photographing process according to the third embodiment Flowchart of photographing process according to the fourth embodiment The figure which shows the example of the image height of the imaging lens Flowchart of shooting process of the fifth embodiment

  Hereinafter, embodiments of an imaging apparatus according to the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
[Overall configuration of imaging device]
FIG. 1 is a block diagram showing an embodiment of an imaging apparatus (digital camera) 10 according to the present invention.

  The digital camera 10 records captured images on a non-volatile storage medium, preferably a memory card 54 or other portable storage medium, and the operation of the entire camera is centrally controlled by a central processing unit (CPU) 40. . Programs executed by the CPU 40 and parameters necessary for the execution, such as the coordinates of the focus area, are stored in a nonvolatile storage medium such as the ROM 53. The CPU 40 uses a volatile storage medium such as a memory (SDRAM) 48 or a VRAM 50 as a buffer when executing a program such as a photographing process.

  The digital camera 10 is provided with an operation unit 38 including a shutter button, a mode dial, a playback button, a MENU / OK key, a cross key, a BACK key, and the like. A signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the digital camera 10 based on the input signal. For example, imaging preparation (zoom lens drive control, AF, AE) and main imaging (for recording) Imaging control including image data acquisition), image processing control, image data recording / reproduction control, display control of the display unit 30, and the like are performed. Note that the operation unit 38 does not need to be configured only with buttons and keys, and can be configured with various user interfaces such as buttons and keys equivalents, for example, a touch panel, an audio sensor, and a line-of-sight sensor.

  The shutter button of the operation unit 38 is an operation button for inputting an instruction to start imaging, and includes a two-stage stroke type switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. . The mode dial selects an imaging mode selection unit that selects any one of an auto imaging mode (normal shooting mode) for capturing a still image, a manual imaging mode, a scene position such as a person, a landscape, a night view, and a moving image mode for capturing a moving image. It is.

  The playback button is a button for switching to a playback mode in which a captured still image or moving image is displayed on the display unit 30 including a liquid crystal monitor or an organic EL (electroluminescence) display. The MENU / OK key is an operation key having both a function as a menu button for instructing to display a menu on the screen of the display unit 30 and a function as an OK button for instructing confirmation and execution of selection contents. It is. The cross key is an operation unit for inputting instructions in four directions, up, down, left, and right, and functions as a button (cursor moving operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. To do. In addition, the up / down key of the cross key functions as a zoom switch at the time of imaging or a playback zoom switch in the playback mode, and the left / right key functions as a frame advance (forward / reverse feed) button in the playback mode. . The BACK key is used to delete a desired object such as a selection item, cancel an instruction content, or return to the previous operation state.

  In the imaging mode, image light indicating the subject is imaged on the light receiving surface of the solid-state imaging device 16 via the imaging lens 12 and the diaphragm 14. Hereinafter, the solid-state imaging device 16 is a CCD, but may be a CMOS or the like. The imaging lens 12 includes a focus lens and a zoom lens. These lenses are driven in the optical axis direction by a lens driving unit 36 controlled by the CPU 40, and focus control for detecting the focus position and moving the focus lens to the detected focus position, from the operation unit 38. In accordance with the instruction, zoom control or the like for changing the focal length to the tele (telephoto) side or the wide (wide angle) side is performed. The diaphragm 14 is composed of, for example, five diaphragm blades, and is driven by the diaphragm driving unit 34 controlled by the CPU 40. For example, the diaphragm 14 is controlled in five stages from the aperture values F2.8 to F11 in increments of 1AV.

  Further, the CPU 40 controls the diaphragm 14 via the diaphragm driving unit 34, and performs charge accumulation time (shutter speed) in the CCD 16, image signal readout control from the CCD 16, and the like via the imaging control unit 32.

  Note that all of the blocks in FIG. 1 do not have to be integrated into one imaging device, and all or part of them may be incorporated into a plurality of independent devices. For example, image data such as the imaging lens 12, the diaphragm 14, the solid-state imaging device 16, the imaging controller 32, the diaphragm driver 34, the lens driver 36, the analog signal processor 18, the A / D converter 20, and the image input controller 22. Each block (imaging unit) related to image acquisition and each block (image processing unit) related to image processing on acquired image data such as the VRAM 50, the memory 48, and the digital signal processing unit 24 are not necessarily configured as an integrated apparatus. May be. The imaging unit may be an external camera unit, and blocks other than the imaging unit may be image processing units that can communicate image signals and the like from the camera.

  FIG. 2 is a diagram showing a configuration example of the CCD 16 according to the present invention. 2A and 2B show an example of a Bayer type arrangement, and FIG. 2C shows an example of a honeycomb type arrangement.

  As shown in FIG. 2A, the CCD 16 has an even line pixel group and an odd line pixel group arranged in a matrix, and photoelectric conversion is performed in each of these two pixel groups. The image signals for the two surfaces can be read independently as will be described later. The plurality of light receiving elements corresponding to each pixel group form an effective pixel region for obtaining an effective imaging signal and an optical black region (hereinafter referred to as “OB region”) for obtaining a black level reference signal. The OB region is actually formed so as to surround the effective pixel region. It suffices if the pixel group for phase difference detection is arranged over the entire effective pixel area, and does not have to be arranged in the OB area.

  As shown in FIG. 2A, the CCD 16 has pixels having R (red), G (green), and B (blue) color filters. In the even lines (0, 2, 4,...), GBGB... Pixels are arranged. On the other hand, RGRRGRG... Pixels are arranged in odd lines (1, 3, 5,...). The pixels on the even lines are alternately shifted to the left or right along the line direction by a half pitch with respect to the pixels on the odd lines.

  The pixels on the even lines are configured as pixels for phase difference detection. That is, as shown in FIG. 3, in the even number line phase difference detection pixels, the center of the photoelectric conversion element (photodiode) PD is shifted from the center of the microlens L1 provided thereabove. On the other hand, in a normal pixel of an odd line, the center of the photoelectric conversion element (photodiode) PD and the center of the microlens L2 provided above the same coincide. The microlens L1 is smaller than the microlens L2.

  In addition, as distinguished by reference signs A and B in FIGS. 2 and 3, the phase difference detection pixel is different from the one in which the microlens L1 is shifted to the left with respect to the photodiode PD (pixel A). , And two types of pixels that are shifted to the right (pixel B). Each of the pixels A and B for phase difference detection includes a color filter of the same color, and preferably includes a G (green) color filter.

  FIG. 4 schematically shows a cross section of the CCD 16. As shown in FIG. 4, the normal pixels are condensed on the photodiode PD by the microlens L2 so as not to be limited in the light receiving direction of the light beam passing through the exit pupil of the imaging lens 12, but the phase difference detection is performed. In the pixel for use, the microlens L1 and the photodiode PD are arranged so as to be limited in the light receiving direction of the light beam passing through the exit pupil of the imaging lens 12, and in the pixel A and the pixel B, the light receiving direction of the light beam. The restriction direction is different.

  Therefore, as shown in FIG. 5, the outputs of the pixel A and the pixel B are out of phase or in phase with each other depending on the state of the rear pin, in-focus, and front pin. Since the phase difference between the output signals of the pixels A and B for detecting the phase difference corresponds to the defocus amount of the imaging lens 12, AF control of the imaging lens 12 can be performed by detecting this phase difference. Although details are omitted, similarly to FIG. 2A, even in the case of the pixel arrangement as shown in FIG. 2B and FIG. 2C, the phase difference detection pixel A including the color filters of the same color and The phase difference can be detected by arranging the centers of the microlenses with respect to the center of the photodiode PD of B to be alternately shifted left or right. Further, the pixels A and B for phase difference detection may be arranged densely or roughly.

  This phase difference is calculated not only for each of the small areas included in the predetermined focus area, but also for each of a plurality of small areas that substantially cover the entire effective pixel area. The plurality of small areas that substantially cover the entire effective pixel area do not need to completely cover the entire effective pixel area, but are densely or roughly distributed over the entire effective pixel area except the focus area. It only has to be arranged. For example, as in Patent Document 8, the effective pixel area is divided into a predetermined unit (for example, 8 × 8 pixels), or less (for example, 1 × 1 pixel) or more (for example, 10 × 10 pixels) in a matrix. A phase difference is calculated for each of the divided areas. Alternatively, the phase difference is calculated for each divided region in a predetermined unit separated from the outer edge of the effective pixel region by a predetermined pitch (for example, one divided region or more or less). In short, it is assumed that the phase difference is calculated over the entire effective pixel area, but it is not necessarily calculated for all of the small areas constituting the effective pixel area.

  Note that the phase difference detection pixels A and B are not limited to the configuration in which the microlens L2 is shifted as shown in FIGS. 3 and 4, and the position of the photodiode PD is shifted with respect to the microlens. Alternatively, as shown in FIG. 6, one microlens L3 may be provided on the two photodiodes PD, and the two photodiodes PD may be used as the pixels A and B, respectively.

  Returning to FIG. 1, the signal charge accumulated in the CCD 16 is read out as a voltage signal corresponding to the signal charge based on the readout signal applied from the imaging control unit 32. The voltage signal read from the CCD 16 is applied to the analog signal processing unit 18 where the R, G, and B signals for each pixel are sampled and held, amplified, and then applied to the A / D converter 20. The A / D converter 20 converts R, G, and B signals that are sequentially input into digital R, G, and B signals and outputs them to the image input controller 22.

  The image signal for one screen (A surface) of even lines is read from the CCD 16, and the image signal for one screen (B surface) of odd lines is subsequently read, and the images of the A and B surfaces are read. The signal is appropriately subjected to image processing as will be described later.

  The digital signal processing unit 24 performs offset processing, white balance correction processing, sensitivity correction processing, gain control processing, gamma correction processing, noise on the A-side and B-side image signals input via the image input controller 22. Obtained by correction processing such as reduction processing, synthesis processing of the image signals of the A and B surfaces after the correction processing, YC processing and YC processing on the composite image that is the image signals of the A and B surfaces subjected to the synthesis processing Edge enhancement for the Y signal (luminance signal) is performed.

  Similar to Patent Document 9, the contour enhancement processing of the digital signal processing unit 24 may include the following processing. In the contour correction process, a contour component is first extracted from the composite image. The extraction of the contour component can be realized, for example, by performing a filtering process such as a band pass filter (BPF) or a high pass filter (HPF) on the luminance signal Y representing the luminance of the composite image. Thereby, contour component data representing the strength of the contour component in each portion (each pixel position) on the original image represented by the image data to be processed is obtained. The intensity of the contour component of each part on the original image represented by the contour component data is a unit area (for example, a 3 × 3 pixel area, or an area of more or less than that, for example, an area of the original image) Depending on the contrast (brightness level difference) within the same divided area as the phase difference calculation unit, the intensity of the contour component is reduced at low contrast areas, and the intensity of the contour component at high contrast areas. growing.

  Next, the digital signal processing unit 24 reads the conversion characteristic information stored in advance in the ROM 53 according to the control from the CPU 40. There are a plurality of types of the conversion characteristic information. Here, the ROM 53 stores first to fourth conversion characteristic information. The conversion characteristic information is information that defines the conversion characteristic by a function, but may be conversion data set in the LUT instead.

  In the first conversion characteristic information, as the value of the input data increases, the slope of the change in the value of the output data with respect to the change in the value of the input data (this slope corresponds to the gain in the intensity conversion of the contour component). A conversion characteristic determined to be small is specified.

  Further, the second conversion characteristic information indicates that the slope of the change in the value of the output data with respect to the change in the value of the input data becomes smaller as the value of the distance difference from the in-focus subject corresponding to the image data in each peripheral region becomes larger. The conversion characteristics determined to be are defined. For example, the second conversion characteristic information includes the intensity (gain) of edge enhancement with respect to image data in the peripheral area = (distance information of the subject in the peripheral area−distance information of the subject in the predetermined focus area) × α (predetermined Constant).

  The third conversion characteristic information defines a conversion characteristic determined such that the slope of the change in the value of the output data with respect to the change in the value of the input data becomes smaller as the value of the input data becomes larger. However, it is assumed that the inclination of the third conversion characteristic information is smaller than the inclination of the first conversion characteristic information, and the edge enhancement strength is weak. The slope of the third conversion characteristic information may be zero.

  In addition, the fourth conversion characteristic information corresponds to the change in the value of the input data as the focal length of the imaging lens 12 corresponding to the image data in the peripheral area becomes smaller (as the angle of view changes from the wide angle side to the telephoto side). A conversion characteristic is defined so that the gradient of change in the value of the output data becomes small.

  The digital signal processing unit 24 converts the strength of the contour component represented by the contour component data based on the read conversion characteristic information. As a result, the degree of contour emphasis (gain) of each part changes according to the strength of the contour component (contrast of each part) in each part of the original image.

  Image data composed of YC signals obtained by the processing of the digital signal processing unit 24 is input to the VRAM 50. The VRAM 50 includes an X area and a Y area each storing image data representing an image for one frame. In the VRAM 50, image data representing an image for one frame is rewritten alternately in the X area and the Y area. Of the X area and Y area of the VRAM 50, the written image data is read from an area other than the area where the image data is rewritten. The image data read from the VRAM 50 is encoded by the video encoder 28 and output to the display unit 30 provided on the back of the camera, whereby the subject image is displayed on the display screen of the display unit 30.

  Further, when the shutter button of the operation unit 38 is pressed (half-pressed) in the first stage, the AF operation and the AE operation are started. That is, the image data of the B surface (the output signal of the phase difference detection pixel) among the image data output from the A / D converter 20 is taken into the AF detection unit 42. The AF detection unit 42 detects the phase difference between the image data of the pixel A and the image data of the pixel B within the predetermined focus area of the image data on the B surface, and outputs information indicating the phase difference to the CPU 40. The CPU 40 obtains the defocus amount based on the phase difference information input from the AF detection unit 42, and determines the position of the focus lens included in the imaging lens 12 via the lens driving unit 36 so that the defocus amount becomes zero. Control.

  However, the AF detection unit 42 detects the phase difference between the image data of the pixel A and the image data of the pixel B in an area other than the predetermined focus area of the image data on the B surface, and sends information indicating the phase difference to the CPU 40. It can also be output.

  The position of the predetermined focus area is set by the CPU 40. For example, when “normal shooting mode” or “landscape mode” is set from the operation unit 38, the CPU 40 sets a position set based on the coordinate information stored in advance in the ROM 53, for example, a rectangular area (for example, a central area of the screen) A 16 × 16 pixel rectangular area surrounding the coordinates of the screen center point) is set as a predetermined focus area. Alternatively, when the “portrait shooting mode” is set from the operation unit 38, the CPU 40 sets the face area detected by the face detection unit 46 as a predetermined focus area. Alternatively, the CPU 40 may set an area appropriately designated by the user via the operation unit 38 as a predetermined focus area. The setting of the predetermined focus area by the CPU 40 may or may not be linked with the selection of the imaging mode via the operation unit 38.

  Face detection by the face detection unit 46 is performed by collating the image of the target area with the face image template (face image feature amount) stored in advance in the ROM 53 while moving the position of the predetermined target area in the captured image. Then, the correlation between the two is examined, and when the correlation score exceeds a preset threshold, the target area is detected as a face area. In addition, as the face detection method, a known method such as a face detection method based on edge detection or shape pattern detection, a face detection method based on hue detection or skin color detection, or the like can be used (see, for example, Patent Document 11).

  The image data output from the A / D converter 20 when the shutter button is half-pressed is taken into the AE detection unit 44. The AE detection unit 44 integrates the G signals of the entire screen, integrates the G signals with different weights in the central portion and the peripheral portion of the screen, or calculates the G signal of the face area detected by the face detection unit 46. Integration is performed and the integrated value is output to the CPU 40. The CPU 40 calculates the brightness of the subject (imaging Ev value) from the integrated value input from the AE detection unit 44, and based on this imaging Ev value, the aperture value of the aperture 14 and the electronic shutter (shutter speed) of the CCD 16 are stored in the ROM 53. Determined according to a predetermined program diagram, controls the aperture 14 via the aperture drive unit 34 based on the determined aperture value, and accumulates charges in the CCD 16 via the imaging control unit 32 based on the determined shutter speed. Control the time.

  When the AE operation and the AF operation are completed and the shutter button is pressed in the second stage (full press), the image data of the A and B planes output from the A / D converter 20 in response to the pressing is obtained. The image is input from the image input controller 22 to the memory 48 and temporarily stored. Hereinafter, acquisition of image data in response to a full press of the shutter button is referred to as main imaging.

  The image data of the A and B surfaces is read from the memory 48 and includes correction processing, image synthesis processing, and luminance data and color difference data generation processing (YC processing) of the synthesized image data in the digital signal processing unit 24. Predetermined signal processing is performed. The YC-processed image data (YC data) is stored in the memory 48 again after being subjected to an edge emphasis process described later in the digital signal processing unit 24. Subsequently, the YC data is output to the compression / decompression processing unit 26, and a predetermined compression process such as JPEG (joint photographic experts group) is executed. The compressed YC data is output and stored again in the memory 48, and then read out by the media controller 52 and recorded in the memory card 54.

[Imaging operation]
Next, the operation at the time of imaging by the digital camera 10 having the above configuration will be described.

  FIG. 7 is a flowchart showing a photographing process of the imaging apparatus (digital camera 10) according to the present invention. This process is controlled by the CPU 40. A program for causing the CPU 40 to execute this processing is stored in a nonvolatile storage medium such as the ROM 53.

  In FIG. 7, the CPU 40 determines whether or not the shutter button of the operation unit 38 is half-pressed (S1 is ON). When S1 of the shutter button is turned ON, the CPU 40 reads out the image data of the B surface including the pixel signal for phase difference detection, and performs the phase difference AF process and the AE process with the read image data. Do.

  That is, in S1, the CPU 40 controls the AE detection unit 44 to perform the integration of the G signal, and determines the imaging conditions such as the shutter speed and the aperture value based on the integrated value.

  In S2, the CPU 40 controls the AF detection unit 42 to detect the phase difference between the image data of the pixel A and the image data of the pixel B in the small area included in the predetermined focus area. A defocus amount in a predetermined focus area is obtained based on the phase difference information.

  In S3, the CPU 40 controls the imaging lens 12 via the lens driving unit 36 so that the main defocus amount of the SDRAM 48 becomes 0, and focuses the imaging lens 12 on a subject in a predetermined focus area. Note that, depending on the arrangement of the imaging lens 12, the main defocus amount of 0 does not necessarily mean a focused state.

  When the imaging lens 12 is brought into focus, the CPU 40 stores phase difference information between the image data of the pixel A and the image data of the pixel B in the predetermined focus area in the SDRAM 48 as main phase difference information.

  Further, when the imaging lens 12 is focused, the CPU 40 controls the AF detection unit 42 so that the image data and the pixel B of the pixel A in the peripheral area, which is a small area included in the effective pixel area other than the predetermined focus area. The phase difference is detected from the image data, and the phase difference information of each peripheral region is stored in the SDRAM 48 as the peripheral phase difference information.

  In S4, when it is determined that the shutter button S2 is turned on, the CPU 40 responds to the shutter speed and aperture value set in the AE process in S1 and the focusing lens position set in the AF process in S2. The main imaging is performed under the imaging conditions.

  In S5, the CPU 40 acquires the image data of the A and B surfaces of the CCD 16 acquired by the main imaging to the VRAM 50, and performs correction processing, composition processing, and YC processing of the image data of these A and B surfaces. The digital signal processing unit 24 is controlled.

  In S <b> 6, the CPU 40 obtains main distance information indicating the distance to the subject in the predetermined focus area from the main phase difference information of the SDRAM 48. Further, the CPU 40 obtains distance information indicating the distance to the subject in each peripheral area from the peripheral phase difference information of the SDRAM 48. This can be calculated based on the fact that the phase difference depends on the distance information as in Patent Document 8.

  The CPU 40 determines, for each peripheral area, the distance difference between the focused subject that is the subject in the predetermined focus area and the peripheral subject that is the subject in each peripheral area from the distance information corresponding to each peripheral area and the main distance information. The distance difference corresponding to each peripheral area is calculated and stored in the SDRAM 48. If the distance difference can be calculated, the method of focusing the imaging lens 12 on the subject in the predetermined focus area is not limited to the TTL phase difference detection method, and an external passive method, an active method, or the like is adopted. Also good. Alternatively, the CPU 40 may calculate the subject distance by a method unrelated to the focusing method. For example, the CPU 40 captures a subject using two or more image capturing units provided at different positions, and supports a plurality of images (a standard image by a standard camera and a reference image by a reference camera) acquired thereby. Corresponding points that are pixels to be searched (stereo matching), calculate the difference (parallax) between the corresponding pixel on the reference image and the pixel on the reference image, and apply the principle of triangulation to the parallax Thus, the distance from the base camera or the reference camera to the point on the subject corresponding to the pixel may be measured.

  Then, the CPU 40 determines whether or not the distance difference corresponding to each peripheral area is less than a predetermined threshold (for example, three times the depth of field, that is, the limit at which blur occurs around the focused subject) for each peripheral area. Judgment. If Yes, the process proceeds to S7. If No, the process proceeds to S8.

  In S <b> 7, the CPU 40 based on the first conversion characteristic information stored in the ROM 53, the intensity of the contour component represented by the contour component data in the predetermined focus area among the YC data after the main imaging stored in the VRAM 50. And the intensity of the contour component represented by the contour component data of the peripheral region where the distance difference is less than a predetermined threshold value, and the digital signal processing unit 24 is controlled so as to perform contour emphasis on these regions.

  In S8, the CPU 40, based on the second conversion characteristic information stored in the ROM 53, out of the YC data after the main imaging, the contour component represented by the contour component data of the peripheral region where the distance difference is equal to or greater than a predetermined threshold. The digital signal processing unit 24 is controlled so as to convert the intensity of the signal and to perform contour enhancement.

  In S9, the CPU 40 controls the compression / decompression processing unit 26 and the media controller 52 so as to record the YC data subjected to the contour emphasis in S7 or S8 on the memory card 54.

  According to the above processing, the degree of contour enhancement is changed based on the difference between the distance to the focused subject in the predetermined focus area and the distance to the subject in each peripheral area other than the predetermined focus area. For this reason, it is possible to prevent excessive outline enhancement from being performed on a subject that is not in focus, and a natural image can be obtained.

Second Embodiment
FIG. 8 is a flowchart showing a photographing process according to the second embodiment. This process is controlled by the CPU 40. A program for causing the CPU 40 to execute this process is stored in the ROM 53.

  S10 to S15 are the same as S1 to S6, respectively. However, if it is determined Yes in S15, the process proceeds to S16, and if it is determined No, the process proceeds to S17.

  S16 is the same as S7.

  In S <b> 17, the CPU 40 determines whether or not the imaging mode set via the operation unit 38 is the person mode. If Yes, the process proceeds to S18, and if No, the process proceeds to S19. The determination in S17 may be performed before S15, and if Yes, the process proceeds to S18. If No, the process may proceed to S16 or S19 after performing S15.

  In S18, the CPU 40, based on the third conversion characteristic information stored in the ROM 53, out of the YC data after the main imaging, the contour component represented by the contour component data of the peripheral area where the distance difference is equal to or greater than a predetermined threshold. The digital signal processing unit 24 is controlled so as to convert the intensity of the signal and to perform contour enhancement. When the inclination of the third conversion characteristic information is 0, contour enhancement for the peripheral region of the YC data after the main imaging is stopped.

  S19 and S20 are the same as S8 and S9, respectively.

  According to the above processing, when the person mode is set, the contour emphasis on the peripheral area close to the person is weakened, so that the contour of the subject person is relatively emphasized.

<Third Embodiment>
FIG. 9 is a flowchart showing a photographing process according to the third embodiment. This process is controlled by the CPU 40. A program for causing the CPU 40 to execute this processing is stored in the ROM 53.

  S21 to S26 are the same as S1 to S6. However, if it is determined Yes in S26, the process proceeds to S27, and if it is determined No, the process proceeds to S28.

  S27 is the same as S7.

  In S28, the CPU 40 determines whether or not the imaging mode set via the operation unit 38 is the landscape mode. If yes, go to S27, if no, go to S29. The determination in S28 may be performed prior to S26, and if Yes, the process proceeds to S27. If No, the process may proceed to S27 or S29 after performing S26.

  S29 and S30 are the same as S8 and S9, respectively.

  According to the above processing, when the landscape mode is set, even if the distance difference is less than a predetermined threshold value, the outline of the set value is enhanced, so that the resolution of the landscape is increased.

<Fourth embodiment>
FIG. 10 is a flowchart showing a photographing process according to the fourth embodiment. This process is controlled by the CPU 40. A program for causing the CPU 40 to execute this processing is stored in the ROM 53.

  S31 to S36 are the same as S1 to S6. However, if it is determined Yes in S36, the process proceeds to S37, and if it is determined No, the process proceeds to S38.

  S37 and S38 are the same as S7 and S8, respectively.

  In S39, the CPU 40 calculates the coordinates of the representative point (such as the center point of the focus area) of the predetermined focus area set by the CPU 40 for the AF operation, and calculates the coordinates of the center point of the YC data. The CPU 40 calculates a displacement distance that is a distance between the coordinates of the representative point and the coordinates of the center point. The CPU 40 determines whether or not the displacement distance is less than a predetermined threshold value. When it determines with Yes, it progresses to S41, and when it determines with No, it progresses to S40. The predetermined threshold is set according to the imaging performance such as lens aberration of the imaging lens 12, and specifically, the image height of the imaging lens 12 is set to 60% (see FIG. 11).

  In S40, the CPU 40 performs the digital signal processing unit 24 so as to perform the contour emphasis processing in which the strength of the contour emphasis of the region at the center of the screen (for example, a 16 × 16 pixel rectangular region surrounding the coordinates of the center point) is lower than that in S38. To control.

  S41 is the same as S9.

  According to the above processing, it is determined whether the representative point of the focus area is near the center of the screen or the peripheral area of the screen, and the degree of contour enhancement changes according to the determination result. That is, since the lens performance at the center of the screen is good, if the focus area is in the peripheral area of the screen, the degree of contour emphasis at the center of the screen is reduced and the entire image is balanced.

<Fifth Embodiment>
FIG. 12 is a flowchart showing a photographing process according to the fifth embodiment. This process is controlled by the CPU 40. A program for causing the CPU 40 to execute this processing is stored in the ROM 53.

  S51 to S59 are the same as S31 to S39.

  In S60, based on the fourth conversion characteristic information stored in the ROM 53, the CPU 40 converts the strength of the contour component represented by the contour component data at the center of the screen among the YC data after the main imaging, and contour enhancement. The digital signal processing unit 24 is controlled to perform the above. That is, the CPU 40 lowers the degree of contour enhancement at the center of the screen below S58 as the shooting angle of view of the imaging lens 12 changes from the wide side to the tele side.

  That is, when the contour emphasis is applied to part or all of the contour component data in the central portion of the screen in S58, contour emphasis lower than the degree of contour emphasis is performed according to the focal length in S60. That is, when a part or all of the central portion of the screen is a peripheral region, contour enhancement to a lower degree is performed on the portion.

  As shown in Patent Document 10, as the focal length (angle of view) of the imaging lens 12 transitions from the wide side to the tele side, the difference between the optical performance at the center of the screen and the optical performance at the periphery of the screen decreases. Optical performance becomes uniform. In this process, when the zoom lens is on the wide side, the degree of contour emphasis at the center of the screen is lower than when it is on the telephoto side. Can take.

  DESCRIPTION OF SYMBOLS 10 ... Imaging device (digital camera), 12 ... Imaging lens, 16 ... Solid-state image sensor, 24 ... Digital signal processing part, 30 ... Display part, 32 ... Imaging control part, 36 ... Lens drive part, 38 ... Operation part, 40 ... Central processing unit (CPU), 42 ... AF detector, 44 ... AE detector, 46 ... Face detector, 48 ... Memory, 54 ... Memory card

Claims (11)

An input unit for inputting image data obtained by photoelectrically converting a subject imaged through an imaging lens by an imaging device;
A distance calculation unit for calculating a distance to the subject;
An edge emphasis processing unit that performs an edge emphasis process according to the distance calculated by the distance calculation unit on the image data input by the input unit;
An image processing apparatus comprising: A focusing control unit that controls the position of the imaging lens so as to focus on a subject existing in a predetermined focusing area of the image data;
The distance calculation unit calculates a difference between a distance to a focused subject that is a subject existing in a predetermined focus area of the image data and a distance to a peripheral subject that is a subject other than the focused subject;
The image processing apparatus according to claim 1, wherein the contour emphasis processing unit performs contour emphasis processing with an intensity corresponding to a difference between a distance to the focused subject and a distance to the peripheral subject on the peripheral subject.   3. The image processing according to claim 2, wherein the contour enhancement processing unit performs contour enhancement processing on the peripheral subject with reduced intensity as a difference between the distance to the focused subject and the distance to the peripheral subject increases. apparatus. It has a mode setting section to set the shooting mode,
The contour emphasis processing unit performs an edge emphasis process on the peripheral subject according to the difference between the distance to the focused subject and the distance to the peripheral subject according to the shooting mode set by the mode setting unit. The image processing apparatus according to claim 2 or 3.   5. The edge enhancement processing unit according to claim 4, wherein the edge enhancement processing unit performs edge enhancement processing with weaker intensity than the focused subject on the peripheral subject in response to the shooting mode set by the mode setting unit being a person mode. Image processing device.   5. The edge enhancement processing unit according to claim 4, wherein the edge enhancement processing unit performs edge enhancement processing of the same intensity as that of the focused subject on the peripheral subject in response to the shooting mode set by the mode setting unit being a landscape mode. Image processing device. A focusing area setting unit for setting a predetermined focusing area in the image data;
A determination unit that determines a difference in position on the image data between a predetermined focusing region set by the focusing region setting unit and a central portion of the image data;
With
The edge enhancement processing unit performs edge enhancement processing with weaker intensity than the region other than the center of the image data in the center of the image data according to the difference determined by the determination unit. An image processing apparatus according to 1.   The image processing apparatus according to claim 7, wherein the contour emphasis processing unit performs contour emphasis processing with an intensity corresponding to a focal length of the zoom lens at a central portion of the image data. The distance calculation unit calculates a distance to the subject based on a defocus amount of a region that substantially covers the entire image data;
The image processing apparatus according to claim 2, wherein the focus control unit controls the position of the imaging lens based on a defocus amount of the predetermined focus area. The image processing device
Inputting image data obtained by photoelectrically converting a subject imaged via an imaging lens by an imaging device;
Calculating a distance to the subject;
Performing the edge enhancement processing of intensity according to the calculated distance on the input image data;
An image processing method for executing.   An image processing program for causing an image processing apparatus to execute the image processing method according to claim 10.

Images