JP2011019245A - Image processing apparatus and photographing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, an image processing method and a photographing apparatus to comparatively easily obtain a captured image emphasizing a photographic subject focused by a photographer without depending on an imaging element and an aperture size of a lens.SOLUTION: An image processing apparatus is constituted of an image processing portion 108 which performs an image process on a first image and generates a third image, an image composing portion 110 which composes the first image and the third image, and an addition-ratio calculating portion 109 which calculates a degree of composition of a composing process between the first image and the third image by the image composing portion 110 based on a difference signal from a difference calculating portion 107.

Description

  The present invention relates to an image processing apparatus and a photographing apparatus capable of acquiring a photographed image in which a subject focused by a photographer is emphasized.

  Currently, photographing devices such as digital cameras and digital videos using solid-state image sensors such as CCD (Charge Coupled Device) sensors and CMOS (Complementary Mental Oxide Semiconductor) sensors are widely used.

  By the way, of the shooting targets, a specific subject is clearly shot, and other subjects except the specific subject are shot as if they are out of focus, and as a result, the specific subject appears to be emphasized and floated as a whole. In order to obtain a so-called “blurred” photographic image, for example, a photographic device having a large size of a solid-state imaging device and a large aperture of a lens for forming a photographic image on the solid-state imaging device is used. There is a need. According to such a type of photographing apparatus, since it is possible to photograph with a sufficiently small depth of field, it is possible to obtain a so-called “blurred” photographed image in which a specific subject is raised. .

  However, when using a photographing device such as a so-called compact type digital camera with a small solid-state image sensor size or lens aperture, the depth of field is sufficiently shallow so that photographing cannot be performed. It was difficult to acquire a captured image. In addition, in any type of photographing device, it is possible to acquire a photographed image in which a specific subject is emphasized by adding a luminance difference or a saturation difference between the specific subject and another subject. It was difficult.

Patent Document 1 listed below discloses an invention for obtaining a blurred image by image processing. According to such image processing, first, one reference image data is separated into two separate and independent image data, that is, image data of the subject area and image data of the background area and output. Then, after performing the blurring process on the separated background image data, the background image data after the blurring process and the reference image data are synthesized. After combining, anti-aliasing processing is performed to prevent the boundary line between the background area and the subject area from becoming jagged and causing a sense of incongruity. An image having “bokeh” can be acquired by such a series of processes. As described above, in Patent Document 1 described below, although a certain “blurred” image can be acquired, the above-described series of complicated processing is required.
JP 2005-229198

  Therefore, the present invention has been made in view of the above-described problems, and image processing in which a captured image in which a specific subject is emphasized can be acquired relatively easily without depending on the size of the aperture of the image sensor or the lens. It is an object to provide an apparatus, an image processing program, an image processing method, and an imaging apparatus.

  An image processing apparatus according to the present invention (1: reference numerals corresponding to the embodiments; the same applies hereinafter) includes a subject output from an imaging means (2) that captures a scene through a focus lens (31) and a diaphragm mechanism (32). An image processing apparatus (1) for processing a field image, focus adjusting means (S107, S305) for adjusting the focus so as to match a specific object in the object field, and focus adjusting means for adjusting a plurality of different aperture amounts Aperture amount setting means (S110, S112, S311, S312) to be set in the aperture mechanism in relation to the adjustment process, and a plurality of output from the imaging means corresponding to the plurality of aperture amounts set by the aperture amount setting means Detection means (104 to 107, S201 to S203, S403 to S404) for detecting the magnitude of the difference in sharpness between the object scene images, and output from the imaging means corresponding to the focus adjusted by the focus adjustment means The detection result of the detection means was referred to the captured object scene image. Comprises an image quality adjusting means for performing the quality adjustment process (108 ~ 110, 113, 116, 119, 122, 125a ~ 125c, 126 ~ 128, S204 ~ S205, S313, S406 ~ S407).

  The imaging means captures the scene through the focus lens and the diaphragm mechanism, and outputs a scene image. The focus adjusting unit adjusts the focus so as to match a specific object in the object scene. The aperture amount setting means sets a plurality of different aperture amounts in the aperture mechanism in relation to the adjustment processing of the focus adjustment means. The detection unit detects the magnitude of the difference in sharpness between the plurality of object scene images output from the imaging unit corresponding to the plurality of aperture amounts set by the aperture amount setting unit. The image quality adjusting means performs image quality adjustment processing with reference to the detection result of the detecting means on the object scene image output from the imaging means corresponding to the focus adjusted by the focus adjusting means.

  The magnitude of the difference in sharpness between a plurality of object scene images respectively corresponding to a plurality of different aperture amounts corresponds to the distance from a specific object to another object in the optical axis direction. The object scene image output from the imaging unit corresponding to the focus suitable for the specific object is subjected to image quality adjustment processing with reference to the magnitude of such difference. Thereby, an object scene image in which a specific object is emphasized can be created.

  Preferably, the detection means includes a high-frequency component extraction means (105, 106, S202, S404) for extracting a plurality of high-frequency components from a plurality of object scene images, and a plurality of high-frequency components extracted by the high-frequency component extraction means. Difference calculation means (107, S203, S404) for calculating the difference between the two is included.

  More preferably, the detection means includes a luminance component extraction means (201) for extracting a plurality of luminance components from the plurality of object scene images, and a plurality of objects based on the plurality of luminance components extracted by the luminance component extraction means. It further includes correction means (202, 203, S201, S403) for correcting the positional deviation between the scene images.

  Preferably, the image quality adjusting means includes specific parameter control means (109, 126) for greatly reducing the specific parameter of the object scene image as the difference increases.

  In one aspect, the image quality adjustment means combines a reduction means (108, 113, 116, 125a to 125c, S204, S313) for reducing specific parameters of the object scene image, and the output of the reduction means and the object scene image. The specific parameter control means controls the composition ratio of the composition means so that the specific parameter of the object scene image is greatly reduced as the difference increases.

  In another aspect, the image quality adjustment means includes a reduction means (113, 116, S204) for reducing the specific parameter of the object scene image, and an increase for increasing the specific parameter of the object scene image in parallel with the reduction processing of the reduction means. Means (119, 122), and combining means (110, S205) for combining the output of the reduction means and the output of the increase means, and the specific parameter control means increases the difference in the specific parameter of the object scene image. The synthesis ratio of the synthesis means is controlled so as to be greatly reduced.

  In another aspect, the image quality adjustment means further includes a reduction means (127) for reducing the specific parameter of the object scene image, and the specific parameter control means increases the reduction amount of the reduction means as the difference increases.

  In one embodiment, the specific parameter includes sharpness.

  In other embodiments, the specific parameter includes brightness.

  In other embodiments, the specific parameter includes saturation.

  Preferably, a first recording means (10, S115) for recording a plurality of object scene images output from the imaging means corresponding to a plurality of aperture amounts set by the aperture amount setting means, and recording by the recording means. And a first reproducing means (10, S117) for reproducing a plurality of object scene images, wherein the detecting means pays attention to the plurality of object scene images reproduced by the first reproducing means, and the image quality adjusting means is the first image quality adjusting means. Attention is paid to one of the plurality of scene images reproduced by the reproducing means.

  More preferably, the first recording means stores a plurality of scene images in a common file.

  Preferably, the image quality adjustment means includes a defocus setting means (S313) for setting the focus to a defocused state, a focus on the object scene image output from the imaging means corresponding to the defocus set by the defocus setting means, and the focus Combining means (127, 110, S407) for synthesizing the object scene image output from the imaging means corresponding to the focus adjusted by the adjusting means is included.

  More preferably, from the imaging unit corresponding to the plurality of object scene images output from the imaging unit corresponding to the plurality of aperture amounts set by the aperture amount setting unit and the defocus set by the defocus setting unit. A second recording means (10, S314) for recording the outputted scene image, and a second reproducing means (10, S402) for reproducing a plurality of scene images recorded by the recording means; The detecting means and the image quality adjusting means pay attention to a plurality of scene images reproduced by the second reproducing means.

  More preferably, the second recording means stores a plurality of scene images in a common file.

  An image processing program according to the present invention is a processor of an image processing apparatus (1) that processes an object scene image output from an imaging means (2) that captures an object scene through a focus lens (31) and an aperture mechanism (32) ( 17), the focus adjustment step (S107, S305) to adjust the focus to match a specific object in the object field, and multiple different aperture amounts are set in the aperture mechanism in connection with the adjustment process of the focus adjustment step Aperture amount setting means (S110, S112, S311, S312), sharpness differences between a plurality of field images output from the imaging means corresponding to a plurality of aperture amounts set by the aperture amount setting step The detection step (S201 to S203, S403 to S404) for detecting the size of the object, and the detection result of the detection step was referred to the object scene image output from the imaging unit corresponding to the focus adjusted by the focus adjustment unit Image quality adjustment process This is an image processing program for executing image quality adjustment steps (S204 to S205, S313, S406 to S407) for performing processing.

  An image processing method according to the present invention is an image processing method for processing an object scene image output from an imaging means (2) that captures an object scene through a focus lens (31) and an aperture mechanism (32). A focus adjustment step (S107, S305) for adjusting the focus so as to match a specific object in the field, and an aperture amount setting means for setting a plurality of different aperture amounts in the aperture mechanism in relation to the adjustment process of the focus adjustment step ( S110, S112, S311, S312), detecting the magnitude of the difference in sharpness between the plurality of field images output from the imaging means corresponding to the plurality of diaphragm amounts set by the diaphragm amount setting step The image quality of the detection step (S201 to S203, S403 to S404) and the image quality adjustment processing that refers to the detection result of the detection step to the object scene image output from the imaging unit corresponding to the focus adjusted by the focus adjustment unit Adjustment Comprising a step (S204 ~ S205, S313, S406 ~ S407).

  According to the present invention, a captured image in which a specific subject is emphasized can be acquired relatively easily.

  The significance or effect of the present invention will become more apparent from the following description of embodiments.

  However, the following embodiment is merely one embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the following embodiment. Absent.

<First Embodiment>

  A first embodiment of the present invention will be described with reference to the drawings. Hereinafter, a description will be given by taking as an example a photographing apparatus such as a digital camera or a digital video camera that performs the photographing method of the present invention. The photographing device may be capable of photographing a moving image as long as it can photograph a still image.

  Hereinafter, “imaging” and “imaging” are used synonymously. Furthermore, the “image signal” may be simply referred to as “image”, but the two are synonymous.

  (Configuration of imaging device)

  FIG. 1 is a block diagram illustrating an internal configuration of the imaging apparatus 1 according to the first embodiment.

  In FIG. 1, an imaging device 1 includes an image sensor (solid-state imaging device) 2 such as a CCD or CMOS sensor that converts incident light into an electrical signal, and a lens unit 3 that forms an optical image of a subject on the image sensor 2. An amplification circuit (not shown) that amplifies an image signal that is an analog electrical signal output from the image sensor 2 and an A / D (Analog / Digital) conversion circuit (FIG. 2) that converts the image signal into a digital image signal. AFE (Analog Front End) 4 provided with a frame memory 5 for temporarily storing digital image signals for a plurality of frames output from the AFE 4 in units of frames, and audio signals input from the outside A microphone (hereinafter referred to as a microphone) 6 that converts the signal, an image processing unit 7 that performs image processing on the image signal from the AFE 4, and an amplifier from the microphone 6. An audio processing unit 8 that converts the audio signal of the log into a digital signal and performs audio processing, and a JPEG (Joint Photographic Experts Group) compression method for the image signal from the image processing unit 7 when shooting a still image, In the case of shooting a moving image, a compression processing unit 9 that performs compression coding processing such as MPEG (Moving Picture Experts Group) compression method on the image signal from the image processing unit 7 and the audio signal from the audio processing unit 8 is provided. .

  In addition, the imaging apparatus 1 includes a driver unit 10 that records the compression-coded signal compressed and encoded by the compression processing unit 9 in an external memory 22 such as an SD card, and a compression encoding that is read from the external memory 22 by the driver unit 10 An expansion processing unit 11 that expands and decodes a signal, an image signal output unit 12 that converts an image signal obtained by decoding by the expansion processing unit 11 into an analog signal, and an image signal converted by the image signal output unit 12 From a display unit 13 having an LCD or the like for displaying an image based on the above, an audio signal output unit 13 for converting a digital audio signal from the decompression processing unit 11 into an analog audio signal and outputting the analog audio signal, and an audio signal output unit 13 Speaker unit 15 for reproducing the analog audio signal.

  In addition, the imaging apparatus 1 includes a TG (Timing Generator) 16 that outputs a timing control signal for matching the operation timing of each block, a CPU (Central Processing Unit) 17 that controls the drive operation of the entire imaging apparatus, A memory 18 for storing each program for each operation and temporarily storing data at the time of program execution, an operation unit 19 for inputting instructions from a user including a shutter button for taking a still image, a CPU 17, and each A bus 20 for exchanging data with the block and a bus 21 for exchanging data between the memory 18 and each block are provided.

  FIG. 2 is an internal configuration diagram of the lens unit 3. The lens unit 3 includes an optical system 35 including a plurality of lenses including a zoom lens 30 and a focus lens 31, a diaphragm 32, and a driver 34. The driver 34 includes a motor or the like for realizing movement of the zoom lens 30 and the focus lens 31 and adjustment of the opening amount of the diaphragm 32 (in other words, the size of the opening portion).

  The imaging device 1 “forms an optical image representing a subject on the imaging surface (also referred to as a light receiving surface) of the image sensor 2 by automatically controlling the position of the focus lens 31” so-called AF ( Auto Focus) function. With this function, ideally, the point on which the optical image representing the subject is formed coincides with the point on the imaging surface of the image sensor 2.

  The AF function is realized, for example, as follows. That is, an AF evaluation value detection unit (not shown) included in the image processing unit 7 extracts a predetermined high frequency component from the luminance signal in the image signal. The CPU 17 controls the position of the focus lens 31 via the driver 34 in accordance with the level (in other words, the size) of the high frequency component, so that an optical image representing the subject is displayed on the imaging surface of the image sensor 2. Make an image.

  Hereinafter, forming an optical image representing the subject on the imaging surface of the image sensor 2 is described as focusing on the subject.

  A state in which an optical image representing the subject is focused (also referred to as a focal point) matches a point on the imaging surface of the image sensor 2 is described as being in focus. However, even if they do not match, if the subject is in a clear range (in other words, if the imaging surface of the image sensor 2 is located within the depth of focus), the subject is in focus. Describe. Conversely, when the subject is out of focus, it is described as out of focus.

  The imaging apparatus 1 can focus on the subject also by so-called manual operation. That is, when the photographer performs a focusing operation on the operation unit 19, the CPU 17 adjusts the position of the focus lens 31 via the driver 34 according to the operation. Thereby, the photographer can focus on a desired subject.

  The imaging apparatus 1 is a so-called AE (Automatic Exposure) function that “automatically controls the aperture of the diaphragm 32, the exposure time of the image sensor 2 and the amplification factor of the AFE 4 to keep the brightness of the captured image substantially constant”. It has.

  The AE function is realized as follows, for example. That is, an AE evaluation value detection unit (not shown) included in the image processing unit 7 integrates luminance signals in all pixels in the image signal output from the AFE 4. Next, the CPU 17 controls the aperture amount of the diaphragm 32 and the exposure time of the image sensor 2 via the driver 34 so that the integrated value (referred to as AE evaluation value) is maintained at a predetermined target value. Alternatively, the amplification factor of the image signal by the AFE 4 is controlled. Thus, the AE function is realized.

  When the optical images incident on the image sensor 2 are the same, the amount of light incident on the image sensor 2 per unit time increases and the value of the luminance signal increases as the aperture amount of the diaphragm 32 increases. If the AE evaluation value is only less than the target value even when the aperture of the diaphragm 32 is maximized, the CPU 17 adjusts the amplification factor of the amplifier circuit of the AFE 12 to keep the AE evaluation value at the target value.

  The imaging apparatus 1 can adjust the opening amount of the diaphragm 32 also by so-called manual operation. That is, when the photographer performs an operation for changing the opening amount of the diaphragm 32 on the operation unit 19 (in other words, an operation for changing the depth of field), the CPU 17 passes the driver 34 in accordance with the operation. The aperture of the diaphragm 32 is adjusted. Thereby, the photographer can shoot with a desired depth of field.

  When the operation for changing the opening amount is an operation for increasing the opening amount, the depth of field becomes shallow, and when the operation for changing the opening amount is an operation for reducing the opening amount, the depth of field becomes deep.

  Note that the image pickup apparatus 1 adjusts the exposure time of the image sensor 2 by the above-described AE function even when the aperture amount of the diaphragm 32 is changed by a photographer's operation, thereby outputting an image output from the AFE 4. The luminance of the signal can be kept constant.

  In the imaging apparatus 1, when an operation for changing the zoom magnification is performed on the operation unit 19 by the photographer, the CPU 17 moves the zoom lens 30 along the optical axis via the driver 34 according to the operation. So-called zoom control is performed. Thereby, the angle of view of photographing by the image sensor 2 is changed (in other words, the image of the subject formed on the imaging surface of the image sensor 2 is enlarged or reduced).

  FIG. 3A and FIG. 3B show two photographed images that are photographed so that the same subject is focused and the depth of field is different. The captured image 100 in FIG. 3A and the captured image 103 in FIG. 3B are obtained by capturing the same subject twice.

  Here, the imaging target is a group of subjects scheduled to be acquired as a captured image.

  Each of the photographed images 100 and 103 is a photographed image representing a subject to be photographed composed of a person 101 and a background 102 such as a building other than the person 101, trees, and a pond.

  The captured images 100 and 103 are both captured images that are captured with the person 101 that is the same subject in focus.

  The photographed image 100 is a photographed image photographed with the aperture amount of the diaphragm 32 adjusted so that the depth of field is relatively shallow. Therefore, in the photographed image 100, the background 102 is out of focus, and the background 102 is unclear and blurred. That is, if the depth of field is shallow, the sharpness of the background 102 decreases.

  On the other hand, the captured image 103 is an image captured by adjusting the aperture amount of the diaphragm 32 so that the depth of field is deeper than that of the captured image 100. Therefore, the photographed image 103 is focused on the background 102, and the background 102 is a clear image. That is, if the depth of field is deep, the sharpness of the background 102 increases.

  In this way, two captured images obtained by capturing the same subject twice, both of which have been captured with a specific subject in focus, and subjects other than the specific subject (ie, subjects other than the specific subject) , Background), one of the images is out of focus and the other is in focus. In this embodiment, “the same subject is in focus and the object field These are referred to as “two photographed images photographed at different depths”.

  In the present embodiment, the two subjects are both taken with a specific subject in focus, and both are out of focus with respect to the background, but the two are different in the degree of defocus. The images are also referred to as “two captured images that are captured so that the same subject is in focus and the depth of field is different”.

  Further, each of a plurality of photographed images that are photographed so that the same subject is focused and the depth of field is different will be simply referred to as “photographed images having different depths of field”.

  The image pickup apparatus 1 in FIG. 1 is configured such that a specific subject from a captured image is more than the background by image processing in the image processing unit 7 even when the size of the image sensor 2 or the aperture of the lens constituting the lens unit 3 is small. An enhanced image (hereinafter referred to as a specific subject enhanced image) can be generated relatively easily. Specifically, the imaging apparatus 1 acquires, for example, the captured image 100 illustrated in FIG. 3A and the captured image 103 illustrated in FIG. 3B by capturing the same subject twice in succession. . Then, a specific subject emphasized image can be generated from the two photographed images.

  Hereinafter, the internal configuration of the image processing unit 7 and the specific subject emphasized image generation process will be described.

  (Internal configuration of image processing unit 7 and specific subject emphasized image generation process)

  FIG. 4 is a diagram for explaining the configuration of the part that performs the specific subject emphasized image generation process and the specific subject emphasized image generation process in the configuration of the image processing unit 7.

  FIG. 4 particularly shows processing for generating a specific subject-emphasized image that has the visual effect that the background 102 appears blurred and the specific subject 101 appears to be floating.

  When the imaging device 1 generates the specific subject emphasized image, two captured images (for two frames) having different depths of field are input to the image processing unit 7 as the input image 1 and the input image 2, respectively.

  In the following, it is assumed that the captured image 100 in FIG. 3A is input as the input image 1 and the captured image 103 in FIG. 3B is input as the input image 2.

  The captured image 100 (hereinafter referred to as the first image 100) and the captured image 103 (hereinafter referred to as the second image 103) are continuously captured by the imaging device 1 and temporarily stored in the frame memory 5. The captured images for two frames output to the image processing unit 7.

  The image processing unit 7 includes an image alignment unit 104 that performs alignment between the first image 100 and the second image 103, and a high frequency component of a luminance signal included in the first image 100 output from the image alignment unit 104 ( Hereinafter, the high frequency component of the luminance signal included in the image is simply referred to as “the high frequency component of the image”.) The second image 103 output from the high frequency component extraction unit 105 and the image alignment unit 104 that extract the image). The high-frequency component of the first image 100 output from the high-frequency component extracting unit 106 and the high-frequency component of the second image 103 output from the high-frequency component extracting unit 106 The difference calculation unit 107 that calculates a difference signal from the component, the image processing unit 108 that performs image processing on the first image 100 to generate the third image 111 illustrated in FIG. 5A, the first image 100, and the first image 100 Combining 3 images 111 Image combining unit 110, and a first image 100 by the image synthesizing unit 110 to synthesize the degree of synthesis process of the third image 111 from the adding ratio calculation unit 109 that calculates based on the difference signal from the difference calculating unit 107.

  When the positional relationship between the captured image and the corresponding pixel is shifted between the first image 100 and the second image 103, the image alignment unit 104 detects this shift and detects the shift in the second image 103. Correct the deviation. Such a positional relationship shift is caused by, for example, camera shake. Note that a shift correction process may be performed on the first image 100. Details of the image alignment processing by the image alignment unit 104 will be described later.

  The high-frequency component extraction unit 105 is configured by, for example, an HPF (High Pass Filter) having a characteristic that the cut-off frequency is 1/10 or more of the Nyquist frequency.

  The high frequency component extraction unit 105 extracts high frequency components for each pixel by performing HPF processing.

  Similarly, the high frequency component extraction unit 106 performs HPF processing on the luminance signal of the second image 103 subjected to the positional deviation correction, and extracts a high frequency component for each pixel.

  The difference calculation unit 107 calculates the difference between the high frequency component of the first image 100 and the high frequency component of the second image 103 (the magnitude of the difference in sharpness between the first image 100 and the second image 103) for each pixel. To calculate.

  As described above, the first image 100 and the second image 103 are both in focus on the person 101. In the first image 100, the background 102 is also in focus, but in the second image 103, the background 102 is not in focus. Therefore, the difference between the high-frequency component of the background 102 image in the first image 100 and the high-frequency component of the background 102 image in the second image 103 is the second high-frequency component of the image of the person 101 in the first image 100 and the second high-frequency component. The value is larger than the difference between the high frequency component of the image of the person 101 in the image 103.

  That is, the difference between the sharpness of the background 102 in the first image 100 and the sharpness of the background 102 in the second image 103 is the sharpness of the person 101 in the first image 100 and the sharpness of the person 101 in the second image 103. Greater than the difference.

  The image processing unit 108 performs image processing on the first image 100. In FIG. 4, the first image 100 is subjected to processing for producing a blurring effect (hereinafter referred to as blurring processing).

  A third image 111 illustrated in FIG. 5A is an image diagram after the image processing unit 108 performs the blurring process on the first image 100.

  For example, as described in pages 108 to 110 of “Digital Image Processing” (second edition, second printing) issued by the CG-ARTS Association, the blurring processing is performed by, for example, the density of adjacent pixel signals. Realized by a smoothing process using an averaging filter or the like that smoothes the change, or as described on pages 131 to 133, the low frequency component remains in the spatial frequency component included in the image signal, and the high frequency This can be realized by LPF (Low Pass Filter) processing that removes components. That is, the third image 111 shown in FIG. 5A is obtained by, for example, subjecting the first image 100 to averaging filter processing and LPF processing.

  Based on the difference result of the difference calculation unit 107, the addition ratio calculation unit 109 derives, for each pixel, an addition ratio K that represents the degree of synthesis of the first image 100 and the third image 103.

  FIG. 6 is a graph of a function representing the relationship between the magnitude of the difference value for each pixel of the high frequency component of the first image 100 and the high frequency component of the second image 103 and the addition ratio K.

  Note that the magnitude of the difference value of the high frequency component corresponds to the “difference degree” of the present invention, and the greater the difference value, the higher the difference degree. The degree of difference is not particularly limited as long as it indicates the degree of difference between the two image signals like the magnitude of the difference value of the high frequency component used in the present embodiment. A correlation function between the high frequency component and the high frequency component of the second image 103 can also be used as the dissimilarity.

  In FIG. 6, the addition ratio K indicates “1” when the magnitude of the difference value of the high frequency component is equal to or less than the threshold value α, and increases linearly as the magnitude of the difference value of the high frequency component becomes larger than the threshold value α. Becomes smaller.

  The addition ratio calculation unit 109 calculates the addition ratio K from the magnitude of the high-frequency component difference value according to the function shown in FIG. 6, but the addition ratio K for the high-frequency component difference value is stored in the image processing unit 7 as a table. It may be prepared in advance.

  The image synthesis unit 110 synthesizes the first image 100 and the third image 111 based on the addition ratio K derived by the addition ratio calculation unit 109 to generate an output image, that is, a specific subject emphasized image.

  A fourth image 112 shown in FIG. 5B is an image diagram of the specific subject emphasized image output from the image composition unit 110.

  Specifically, the image composition unit 110 generates a fourth image 112 for each pixel as an output image by a weighted addition process according to the following formula (1).

Fourth image 112 = (first image 100 × addition ratio K)
+ (Third image 111 × (1−addition ratio K)) (1)

  For the image of the person 101 in the first image 100 and the image of the person 101 in the third image 103, the difference value of the high frequency component is small. For this reason, the pixel addition ratio K corresponding to the image of the person 101 is “1” or a value close to “1”. Therefore, according to Equation 1, the image of the person 101 in the fourth image 112 is substantially equal to the image of the person 101 in the first image 100.

  On the other hand, for the image of the background 102 in the first image 100 and the image of the background 102 in the second image 103, the addition ratio K approaches zero as the difference value of the high frequency component becomes larger than the threshold value α. Accordingly, the image of the background 102 in the third image 111 is mixed with the image of the background 102 in the first image 100, and the image of the background 102 in the fourth image 112 is more blurred than the image of the background 102 in the first image 100. The image is highly effective.

  As a result, the background 102 in the fourth image 112 is an image that is more blurry than the background 102 in the first image 100, and the fourth image 112 is an image in which the person 101 appears to stand up.

  In the above description, the difference between the high frequency component of the first image 100 and the high frequency component of the second image 103 after positional deviation correction is calculated. However, since it is only necessary to calculate the degree of difference between the high frequency components of the first image 100 and the second image 103 after the positional deviation correction, instead of calculating the difference, the high frequency components of the first image 100 and the positional deviation correction are performed. It is also possible to calculate the ratio between the subsequent high frequency components of the second image 103.

  In the above description, the image processing unit 108 blurs the first image 100 to generate the third image 111, and the image synthesis unit 110 synthesizes the first image 100 and the third image 111 to generate the fourth image. 112 is generated. However, an image that produces the same effect can be obtained by synthesizing the second image 103 and the third 'image by setting the image generated by performing the blurring process on the second image as a third' image (not shown). can do.

  Here, in FIG. 6, the threshold value α of the addition ratio K is obtained after the combination process when the difference value of the high frequency component is reflected in the addition ratio K that is the degree of combination of the first image 100 and the third image 111. This is determined from the viewpoint of whether the viewer can recognize the influence of the difference value on the fourth image 112. That is, the threshold value α is the maximum value at which the viewer cannot recognize the influence of the difference value on the fourth image 112.

  Similarly, the gradient (gradient) of the graph of FIG. 6 is also determined from the viewpoint of whether the viewer can recognize the degree of change in the difference value on the fourth image 112.

  Therefore, FIG. 5 shows a function that decreases linearly as the threshold value α is exceeded. However, the function may be a function that decreases gradually as the difference value of the high frequency component increases, even if it is not linear.

  (Internal configuration of image registration processing unit 104 and registration processing)

  FIG. 7 is a diagram illustrating the internal configuration of the image registration processing unit 104 and the image registration processing.

  The image alignment processing unit 104 extracts a luminance signal from each of the input first image 100 and second image 103, and the first image 100 and the second image based on the extracted luminance signal. 103, and a misalignment correcting unit 203 that performs misalignment correction processing on the first image 100 based on the detected misalignment.

  For example, when each pixel of the input image has signal values IR, IG, and IB according to the RGB format, the luminance signal extraction unit 201 calculates and outputs the magnitude Y of the luminance signal at each pixel by the following formula (2). .

  Y = 0.299IR + 0.587IG + 0.114IB (2)

  When the luminance signal of the first image 100 and the luminance signal of the second image 103 are input, the positional deviation detection unit 202, for example, based on the representative point matching method that is a well-known method, A motion vector with respect to the image 103 is detected.

  When the motion vector between the first image 100 and the second image 103 calculated by the misalignment detection unit 202 is M, the misregistration correction unit 203 cancels the motion vector M over the entire pixels of the first image 100. Move.

  FIG. 8 and FIG. 9 are diagrams for explaining the misalignment correction processing by the misalignment correcting unit 203.

  FIG. 8 shows the horizontal and vertical components of the motion vector M between the first image 100 and the second image 103.

  In FIG. 8, it is assumed that the motion vector M generated between the first image 100 and the second image 103 is M (Xm, Ym). FIG. 9 shows that the position of the image corresponding to the pixel P (X, Y) in the first image 100 has moved to the pixel P ′ (X + Xm, Y + Ym) in the second image 103 due to the generation of the motion vector M. ing.

  Accordingly, the positional deviation correction unit 203 moves the pixel position P ′ (X + Xm, Y + Ym) on the second image 103 to the pixel position P (X, Y) on the first image 100.

  That is, the luminance signal of the second image 103 is converted so that the luminance value at the pixel position P ′ (X + Xm, Y + Ym) on the second image 103 becomes the luminance value at the pixel position P (X, Y). Deviation correction is performed.

  The imaging apparatus 1 has a specific subject emphasized photographing mode for generating a specific subject emphasized image as described above in addition to a normal still image photographing mode in still image photographing.

  When the photographer shoots in the specific subject shooting mode, the imaging apparatus 1 acquires a plurality of captured images having different depths of field by continuously shooting while changing the depth of field, and these fields of field. A specific subject emphasized image is generated from a plurality of photographed images having different depths. The acquired plural captured images having different depths of field and the generated specific subject emphasized image are recorded in the external memory 22.

  Specifically, when a photographing operation is performed on the operation unit 19 by the photographer, the imaging apparatus 1 focuses on the subject (person 101) focused on by the photographer using the AF function. Subsequently, the aperture of the diaphragm 32 is adjusted by the AE function, the exposure time (shutter speed) of the image sensor 2 is adjusted, and the hue is adjusted by the AWB (Auto White Balance) function, and then shooting (first shooting) is performed. Do.

  Here, the AWB function means that the imaging device 1 determines a light source of light irradiated on a subject, automatically determines a white hue according to the light source, and other colors according to the white hue. This is a function for determining the hue.

  Following the first shooting, the imaging apparatus 1 adjusts the opening amount of the diaphragm 32 so that the depth of field is deeper than the first shooting, and performs shooting (second shooting).

  Note that how much the depth of field differs between the first shooting and the second shooting can be set in advance by the photographer according to his / her preference.

  After the first shooting and the second shooting, the imaging apparatus 1 has an image obtained by the first shooting (the shot image 100 in FIG. 3A) and an image obtained by the second shooting (FIG. 3B). Based on the photographed image 103 of FIG. 5, a specific subject emphasized image (fourth image 112 in FIG. 5B) is generated, and a total of three images are recorded in the external memory 22.

  Note that the shooting order of the first shooting and the second shooting may be reversed.

  Further, in order to obtain an image having a different depth of field from the image obtained by the first photographing, the third, fourth,... And plural times of photographing are performed in addition to the second photographing. Also good. In this case, the CPU 17 selects two images that are stored in the frame memory 5 and that are suitable for generating the specific subject emphasized image from among the captured images having different depths of field.

  (Shooting operation by the imaging apparatus 1)

  Next, a method for acquiring an image in which a specific subject is emphasized using the imaging apparatus 1 according to the present embodiment will be described.

  In FIG. 10, the imaging apparatus 1 acquires the first image 100 (the captured image 100 in FIG. 3A) and the second image 103 (the captured image 103 in FIG. 3B), and the specific subject is emphasized. 6 is a flowchart showing a series of processes until a fourth image 112 in FIG. 5B is generated.

  A control program corresponding to these flowcharts is stored in a flash memory (not shown). The CPU 17 is always involved in the operation of each step described below.

  When the power of the imaging apparatus 1 is turned on, the process proceeds to step S101.

  In step S101, the photographer selects an operation mode of the imaging apparatus 1. The imaging apparatus 1 has a shooting mode for shooting a moving image and a still image and a playback mode for playing back a shot image that has already been shot and recorded in the external memory 22.

  Here, it is assumed that the photographing mode is selected by the photographer and that the specific subject emphasizing photographing mode is selected.

  In step S102, the imaging apparatus 1 shifts to the preview mode.

  In the preview mode, an analog image signal obtained by photoelectric conversion of the image sensor 2 via the lens unit 3 is converted into a digital image signal by the AFE 4 and output to the image processing unit 7. In the image processing unit 7, the digital image signal is subjected to image processing such as white balance processing, and is displayed on the display unit 13 through the image signal output unit 12.

  In step S103, the composition of the subject and the zoom magnification are adjusted in response to the operation of the photographer.

  In step S104, the CPU 17 determines whether or not the shutter button of the operation unit 19 is half-pressed.

  The operation unit 19 of the imaging apparatus 1 is provided with a shutter button (not shown) for still image shooting. The shutter button is a two-stage switch, and when the photographer pushes the shutter button almost halfway, the first switch is turned on. When the shutter button is pushed down to the end (hereinafter, pushing the shutter button to the end is referred to as “full press”), the second switch is turned on.

  If it is determined that the shutter button is half-pressed, the process proceeds to step S105. Otherwise, the process returns to step S102, and the imaging apparatus 1 continues the preview mode. In step S105, the first shooting condition (normal shooting condition) is set for the shooting target. In the setting of the first shooting condition, the imaging apparatus 1 focuses on the subject (person 101) focused on by the photographer by the AF function, sets the aperture amount of the diaphragm 32, and sets the exposure time of the image sensor 2 by the AE function. Shutter speed) and hue by AWB function are set.

  In step S106, the CPU 17 of the imaging apparatus 1 determines whether the current still image shooting mode is the normal shooting mode or the specific subject emphasized shooting mode. If it is the specific subject emphasis shooting mode, the process proceeds to step S107, and if not, the process proceeds to step S108.

  In step S107, the imaging apparatus 1 sets a second imaging condition (small aperture imaging condition) for the imaging target. In the setting of the second shooting condition, the imaging apparatus 1 sets the aperture amount of the diaphragm 32 to be smaller than that of the first shooting condition, that is, the depth of field is deepened.

  As for the second imaging condition, the photographer can set in advance how much the aperture amount of the diaphragm 32 is reduced from the first imaging condition. Thereby, the photographer can change the degree of emphasis of the person 101 according to his / her preference.

  In step S108, the CPU 17 determines whether or not the shutter button has been fully pressed. In step S109, the CPU 17 determines whether or not the operation of the shutter button has been released.

  If it is determined that the shutter button has been fully pressed, the process proceeds to step S110. If it is determined that the operation of the shutter button has been released, the process returns to step S102.

  In step S <b> 110, the imaging apparatus 1 captures the scene with the first imaging condition set in step S <b> 105, and stores the first image 100 acquired thereby in the frame memory 5.

  In step S111, the CPU 17 of the imaging apparatus 1 determines whether the current still image shooting mode is the normal shooting mode or the specific subject emphasized shooting mode. If it is the specific subject emphasis shooting mode, the process proceeds to step S112, and if not, the process proceeds to step S114.

  In step S <b> 112, the imaging apparatus 1 shoots the scene under the second shooting condition set in step S <b> 107 and stores the second image 103 acquired thereby in the frame memory 5.

  In step S113, the specific subject emphasized image generation process is performed by the image processing unit 7, and the fourth image 112 that is the specific subject emphasized image is generated. When the process is completed, the process proceeds to step S114.

  The specific subject emphasized image generation process will be described later.

  In step S <b> 114, under the control of the CPU 17, the image processing unit 7 performs image processing on the captured image or the captured image and the specific subject emphasized image, and then the compression processing unit 9 performs compression processing on the external memory 22. Stored. Then, the process returns to step S102.

  (Specific subject enhancement image generation processing)

  FIG. 12 is a flowchart illustrating a series of processing until the imaging apparatus 1 generates the fourth image 112 from the first image 100 and the second image 103.

  In step S <b> 201, a positional deviation between the first image 100 and the second image 103 output from the frame memory 5 is detected, and when there is a positional deviation, a positional deviation correction process is performed on the second image 103. Thereafter, the process proceeds to step S202.

  In step S202, the high frequency components of the first image 100 and the second image 103 after the positional deviation correction are calculated. Then, it progresses to step S203.

  In step S203, the difference between the high frequency component of the first image 100 and the high frequency component of the second image 103 after the positional deviation correction is calculated, and the addition ratio K is derived with reference to the calculated difference.

  In step S204, the first image 100 is subjected to blurring processing to generate a third image 111 in FIG. Thereafter, the process proceeds to step S205.

  In step S205, the first image 100 and the third image 111 are weighted and added according to the addition ratio K, thereby generating a fourth image 112 in which the person 101 is emphasized.

As described above, by performing the processing shown in FIGS. 10 to 12, the photographer blurs the background 102 and emphasizes the person 101, and as a result, the “blurred taste” that makes the person 101 stand out. A fourth image 112 can be acquired.
<Second Embodiment>

  Next, a second embodiment will be described. FIG. 13 is a diagram for explaining the configuration of the part that performs the specific subject emphasized image generation process and the specific subject emphasized image generation process in the configuration of the image processing unit 7 according to the second embodiment.

  In the second embodiment, a method is shown in which a process for reducing the luminance of the background 102 is performed, and as a result, the luminance of the person 101 becomes relatively high, and a captured image in which the person 101 appears to stand up is obtained.

  In FIG. 13, parts denoted by the same reference numerals as those in FIG. 4 are parts having the same functions and operations as those in the first embodiment, and thus description of the functions and operations is omitted.

  The image processing unit 113 performs processing (hereinafter, referred to as luminance reduction processing) that has an effect of reducing luminance on the first image 100 in FIG.

  The luminance value in each pixel of the first image 100 can be expressed by the above-described equation (2).

  The image processing unit 113 performs a process for reducing the luminance at a certain rate for each pixel of the first image 100 as the luminance reduction process.

  A fifth image 114 illustrated in FIG. 14A is an image diagram of an image generated by performing the luminance reduction process on the first image 100.

  The image synthesizing unit 110 synthesizes the first image 100 and the fifth image 114 by the weighted addition process of Expression (3) based on the addition ratio K derived by the addition ratio calculation unit 109, as shown in FIG. A sixth image 115 is generated as an output image.

Sixth image 115 = (first image 100 × addition ratio K)
+ (5th image 114 × (1−addition ratio K)) (3)

  As a result, the background 102 in the sixth image 115 is an image having a reduced luminance than the background 102 in the first image 100. As a result, an image in which the person 101 appears to float is obtained.

  Next, a flowchart for acquiring the sixth image 115 will be described.

  FIG. 12 shows a flowchart for acquiring the fourth image 112 in the first embodiment. The case of acquiring the sixth image 115 of the second embodiment will be described with reference to FIG. In this case, in step S204 of the flowchart of FIG. 12, instead of the blurring process on the first image 100, a process of “performing the luminance reduction process on the first image 100 to generate the fifth image 114” is executed. .

  In the above description, the image processing unit 113 performs the luminance reduction processing on the first image 100 to generate the fifth image 114 with reduced luminance, and the image composition unit 110 performs the first image 100 and the fifth image 114. Is synthesized. However, the image processing unit 113 performs luminance reduction processing on the second image 103 to generate a 5 ′ image (not shown), and the image composition unit 110 combines the 5 ′ image and the second image 103. However, an image having the same effect can be acquired.

Accordingly, the photographer can acquire the sixth image 115 in which the brightness of the background 102 is reduced and the person 101 is emphasized and appears to float.
<Third Embodiment>

  Next, a third embodiment will be described. FIG. 15 is a diagram for explaining the configuration of the part that performs the specific subject emphasized image generation process and the specific subject emphasized image generation process in the configuration of the image processing unit 7 according to the third embodiment.

  In the third embodiment, a process of reducing the saturation of the background 102 is performed, and as a result, a captured image in which the saturation of the person 101 is relatively high with respect to the background 102 and the person 101 appears to float is acquired. How to do.

  In FIG. 15, the parts denoted by the same reference numerals as those in FIG. 4 are parts having the same functions and operations as those in the first embodiment, and thus description of the functions and operations is omitted.

  The image processing unit 116 performs a process (hereinafter, referred to as a saturation reduction process) having an effect of reducing the saturation on the first image 100.

  Here, the saturation means the vividness of the color and the purity of the color. When the saturation is high, the color approaches a pure color, and when the saturation is low, the color approaches a cloudy color (gray). The range of the saturation value is 0 to 100%. FIG. 16 shows a hue circle diagram. A hue ring is a ring in which hues are arranged in a ring shape. In the figure, the saturation is expressed by the length of the arrow. That is, assuming that the color difference signals of each pixel in the first image 100 are RY and BY, the saturation of each pixel in the first image 100 is calculated according to Equation (4).

Therefore, the image processing unit 113 performs a process of reducing the saturation of each pixel at a certain rate for each pixel of the first image 100 as the saturation reduction process.

  A seventh image 117 illustrated in FIG. 17A is an image diagram of an image generated by performing saturation reduction processing on the first image 100.

  The image synthesizing unit 110 synthesizes the first image 100 and the seventh image 117 by the weighted addition processing of Expression (5) based on the addition ratio K derived by the addition ratio calculation unit 109, as shown in FIG. The eighth image 118 is generated as an output image.

Eighth image 118 = (first image 100 × addition ratio K)
+ (7th image 117 × (1−addition ratio K)) (5)

  As a result, the background 102 in the eighth image 118 is an image with reduced saturation than the background 102 in the first image 100. As a result, an image in which the person 101 appears to float is obtained.

  Next, a flowchart for acquiring the eighth image 118 will be described.

  FIG. 12 shows a flowchart for acquiring the fourth image 112 in the first embodiment. The case where the eighth image 118 of the third embodiment is acquired will be described with reference to FIG. In this case, in step S204 in the flowchart of FIG. 12, instead of the blurring process for the first image 100, a process of “saturating the first image 100 and generating a seventh image 117” is executed. The

  In the above description, the image processing unit 113 performs saturation reduction processing on the first image 100 to generate the seventh image 117 with reduced saturation, and the image composition unit 110 performs the first image 100 and the seventh image reduction. The image 117 is synthesized. However, the image processing unit 113 performs luminance reduction processing on the second image 103 to generate a seventh ′ image (not shown), and the image composition unit 110 combines the seventh ′ image and the second image 103. Even in this case, an image having the same effect can be acquired.

As a result, the photographer can acquire the eighth image 118 in which the saturation of the background 102 is reduced and the person 101 appears to be highlighted.
<Modification of Second Embodiment>

  FIG. 18 is a diagram for explaining a modification of the second embodiment.

  In FIG. 18, the parts denoted by the same reference numerals as those in FIG. 4 are parts having the same functions and operations as those in the first embodiment, and thus description of the functions and operations is omitted.

  In FIG. 18, the image processing unit 119 performs a process (hereinafter, referred to as a brightness increase process) that has an effect of increasing the brightness on the first image 100.

  A ninth image 120 shown in FIG. 19A is an image diagram when the luminance increasing process is performed on the first image 100.

  Based on the addition ratio K derived by the addition ratio calculation unit 109, the image composition unit 110 calculates the ninth image 120 and the fifth image 114 on which the luminance reduction processing illustrated in FIG. The tenth image 121 shown in FIG. 19B is generated as an output image.

10th image 121 = (9th image 120 × addition ratio K)
+ (5th image 114 × (1−addition ratio K)) (6)

  As a result, the person 102 in the tenth image 121 is an image with higher brightness than the person 102 in the first image 100. As a result, an image in which the person 101 appears to float is obtained.

  Next, a flowchart for acquiring the tenth image 121 will be described.

  FIG. 12 shows a flowchart for acquiring the fourth image 112 in the first embodiment. With reference to FIG. 12, a case of acquiring the tenth image 121 in the modification of the second embodiment is shown. explain. In this case, in step S204 in the flowchart of FIG. 12, instead of the blurring process for the first image 100, “the first image 100 is subjected to the luminance reduction process and the luminance increase process, and the fifth image 114 and the ninth image 120 are processed. Is generated. Furthermore, in step S205, a process of “weighted addition of the fifth image 114 and the ninth image 120” is executed.

  In the above description, the image processing units 113 and 119 perform the brightness reduction process and the brightness increase process on the first image 100, respectively, and generate the fifth image 114 and the ninth image 120, respectively. In addition, the image synthesis unit 110 synthesizes the fifth image 114 and the ninth image 120. However, the image processing units 113 and 119 perform a brightness reduction process and a brightness increase process on the second image 103, respectively, to generate a 5 ′ image (not shown) and a 9 ′ image (not shown), respectively. The image composition unit 110 may compose the fifth 'image and the ninth' image. As a result, an image having the same effect can be acquired.

As a result, the photographer can acquire the tenth image 121 in which the brightness of the person 101 is increased, and as a result, the person 101 appears to stand out.
<Modification of Third Embodiment>

  FIG. 20 is a diagram for explaining a modification of the third embodiment.

  In FIG. 20, the portions denoted by the same reference numerals as those in FIG. 4 are portions having the same functions and operations as those in the first embodiment, and thus description of the functions and operations is omitted.

  In FIG. 20, the image processing unit 122 performs a process (hereinafter referred to as “saturation increasing process”) that produces an effect of increasing the saturation on the first image 100.

  An eleventh image 123 shown in FIG. 21A is an image diagram when the saturation increase process is performed on the first image 100.

  Based on the addition ratio K derived by the addition ratio calculation unit 109, the image composition unit 110 calculates the formula (7) from the eleventh image 123 and the seventh image 117 on which the saturation reduction processing illustrated in FIG. ) To generate a twelfth image 124 shown in FIG. 21B as an output image.

12th image 124 = (11th image 123 × addition ratio K)
+ (7th image 117 × (1−addition ratio K)) (7)

  As a result, the person 102 in the twelfth image 124 is an image with higher saturation than the person 102 in the first image 100. As a result, an image in which the person 101 appears to float is obtained.

  Next, a flowchart for acquiring the twelfth image 124 will be described.

  FIG. 12 shows a flowchart for acquiring the fourth image 112 in the first embodiment. With reference to FIG. 12, a case where the twelfth image 124 in the modification of the third embodiment is acquired is shown. explain. In this case, in step S204 in the flowchart of FIG. 12, instead of the blurring process on the first image 100, “saturation reduction processing and saturation increase processing are performed on the first image 100, and the seventh image 117 and eleventh image are processed. The “generate image 123” process is executed. Further, in step S205, a process of “weighted addition of the seventh image 117 and the eleventh image 123” is executed.

  In the above description, the image processing units 116 and 122 perform the saturation reduction process and the saturation increase process on the first image 100, respectively, and generate the seventh image 117 and the eleventh image 123, respectively. Further, the image composition unit 110 synthesizes the seventh image 117 and the eleventh image 123. However, the image processing units 116 and 122 perform a saturation reduction process and a saturation increase process on the second image 103, respectively, to obtain a seventh 'image (not shown) and an eleventh' image (not shown). Each of the images may be generated, and the image combining unit 110 may combine the seventh 'image and the eleventh' image. As a result, an image having the same effect can be acquired.

  As a result, the photographer can acquire the twelfth image 124 in which the saturation of the person 101 is increased and as a result, the photographer appears to stand out.

  As described above, according to the embodiment of the present invention, the imaging device 1 acquires the input image 1 and the input image 2 as two captured images having different depths of field, and the acquired two captured images are obtained. Input to the image processing unit 7. For example, the image processing unit 7 performs processing for reducing the level of the image signal (the magnitude of the image signal) of the input image 1 such as blurring processing, luminance reduction processing, and saturation reduction processing on the input image 1. . Then, the image processing unit 7 calculates the degree of difference between the input image 1 and the input image 2 and performs processing for reducing the image signal level for the input image 1 and the input image 1 according to the degree of difference. The applied image is added for each pixel. Specifically, the higher the difference, the higher the ratio of the images that have undergone the process of reducing the image signal level. As a result, only the background 102 can be blurred, the luminance can be reduced, or the saturation can be reduced. As a result, a person 101 can be emphasized and a photographed image that appears to float can be acquired.

  Further, as a modification of the embodiment of the present invention, for example, the input image 1 is subjected to processing for reducing the level of the image signal of the input image 1 such as luminance reduction processing or saturation reduction processing, Processing for increasing the level of the image signal of the input image 1 such as saturation increase processing is also performed.

  Then, the image processing unit 7 determines the image signal level for the input image 1 and the image that has been subjected to the processing for increasing the image signal level according to the difference between the input image 1 and the input image 2. And the image that has been subjected to the processing for reducing the image are added for each pixel. Specifically, the higher the difference, the higher the ratio of the images that have undergone the process of reducing the image signal level. On the contrary, the lower the difference, the higher the ratio of the images that have been processed to increase the image signal level. As a result, only the background 102 can be blurred, the luminance can be reduced, or the saturation can be reduced. As a result, a person 101 can be emphasized and a photographed image that appears to float can be acquired.

  In the above embodiment, blur processing, luminance reduction processing, and saturation reduction processing have been described as examples of processing for reducing the level of the image signal. However, any processing that can reduce the level of the image signal is described. The present invention is not limited to these, and two or all of the blur processing, luminance reduction processing, and saturation reduction processing may be executed simultaneously.

  Similarly, although the luminance increasing process or the saturation increasing process has been described as the process for increasing the level of the image signal, the process is not limited to this as long as the process can increase the level of the image signal.

  In the above-described embodiment, the specific subject emphasized image generation process is executed under the shooting mode. However, the specific subject emphasized image generation process may be executed under the reproduction mode. In this case, the flowchart shown in FIG. 11 needs to be partially corrected as shown in FIG. 22, and further, in the reproduction mode, the process needs to be executed according to the flowchart shown in FIG.

  Referring to FIG. 22, when the process of step S112 is completed, the process proceeds to step S115. In step S115, an MPF (Multi Picture Format) file storing the normal aperture image obtained by the normal imaging process and the small aperture image obtained by the small aperture imaging process is created, and the created MPF file is stored in the external memory 22. To record. When the process of step S115 is completed, the process returns to step S102.

  Referring to FIG. 23, an MPF file recorded in external memory 22 is designated in step S116, and a normal aperture image and a small aperture image are reproduced from the designated MPF file in step S117. In step S113, the specific subject emphasized image generation process is executed while paying attention to the reproduced normal aperture image and small aperture image. The composite image generated thereby is displayed in step S118.

  In the above-described embodiment, the blurring amount by the image processing unit 108 shown in FIG. 4, the luminance reduction amount by the image processing unit 113 shown in FIG. 13, and the saturation reduction amount by the image processing unit 116 shown in FIG. Both are uniform. However, the blurring amount, the luminance reduction amount, or the saturation reduction amount may be varied, and images having different blurring amounts, luminance reduction amounts, or saturation reduction amounts may be combined with the input image 1. In this case, the image processing unit 7 needs to be configured as shown in FIG.

  According to FIG. 24, the input image 1 is given in parallel to the image processing units 125a to 125c. The image processing units 125a to 125c execute blurring processing according to different blurring amounts, luminance down processing according to different luminance reduction amounts, or saturation reduction processing according to different saturation reduction amounts. The image composition unit 110 combines the input image 1 and the images output from the image processing units 125 a to 125 c with reference to the output of the addition ratio calculation unit 109.

  Further, in the image processing unit 7 shown in FIG. 4, the blurring process is performed by the image processing unit 108, while the composition process is performed by the image composition unit 110. However, as shown in FIG. 25, the image processing unit 108 and the image synthesis unit 110 are replaced with the LPF 127, the addition ratio calculation unit 109 is replaced with the cutoff frequency calculation unit 126, and the cutoff characteristic of the LPF 127 is calculated as the cutoff frequency. The control may be performed based on the cutoff frequency calculated by the unit 126.

In this case, the cut-off frequency changes as shown in FIG. That is, the cutoff frequency decreases in accordance with the decrease in the difference signal calculated by the difference calculation unit 107. In the range where the magnitude of the difference signal is lower than the threshold value β, the cutoff frequency is set to “Fc”. As a result, a composite image in which the specific subject is emphasized is output from the LPF 127.
<Fourth Embodiment>

  Next, a fourth embodiment will be described. FIG. 27 is a diagram for explaining the configuration of the part that performs the specific subject emphasized image generation process and the specific subject emphasized image generation process in the configuration of the image processing unit 7 according to the first embodiment.

  In the fourth embodiment, in addition to the normal aperture image obtained under the normal imaging conditions and the small aperture image obtained under the small aperture imaging conditions, the defocus imaging conditions (imaging conditions that are extremely out of focus) are set. The defocused image obtained below is used.

  In FIG. 27, the parts denoted by the same reference numerals as those in FIG. 4 are parts having the same functions and operations as those in the first embodiment, and thus the description of the functions and operations is omitted.

  The image alignment unit 128 has the same configuration as the image alignment unit 104, and performs alignment between the normal aperture image and the defocused image. The normal aperture image and the defocused image output from the image alignment unit are synthesized by the image synthesis unit 110. In the synthesis process, the addition ratio K calculated by the addition ratio calculation unit 109 is referred to.

  Next, a method for acquiring an image in which a specific subject is emphasized using the imaging apparatus 1 according to the present embodiment will be described.

  28 to 29 are flowcharts showing a series of processes until the imaging apparatus 1 acquires the normal aperture image, the small aperture image, and the defocused image, and records these acquired images in the external memory 22. FIG. 30 is a flowchart showing a series of processing until a composite image in which a specific subject is emphasized based on a normal aperture image, a small aperture image, and a defocus image recorded in the external memory 22 is displayed. .

  A control program corresponding to these flowcharts is stored in a flash memory (not shown). The CPU 17 is always involved in the operation of each step described below.

  Referring to FIG. 28, in step S301, the operation mode is set to the preview mode, and in step S302, the zoom magnification and the aperture amount are controlled in response to the photographer's operation. In step S303, AF control, AE control, and AWB control are executed. In step S304, it is determined whether or not the shutter button is half-pressed. If NO in step S304, the process returns to step S302, and if YES in step S304, the process proceeds to step S305.

  In step S305, normal imaging conditions (first imaging conditions) are set in the same manner as in step S105 shown in FIG. In step S306, small aperture imaging conditions (second imaging conditions) are set in the same manner as in step S107 shown in FIG. In step S307, a defocus imaging condition (third imaging condition) that sets the focus far from the subject is set.

  In any of steps S305 to S307, the focus is adjusted to a common subject (specific subject) in the object scene. The aperture amount set in step S307 is the same as the aperture amount set in step S305.

  In step S308, it is determined whether or not the shutter button has been fully pressed. In step S309, it is determined whether or not the operation of the shutter button has been released. If “YES” in the step S308, the process proceeds to a step S310, and if “YES” in the step S309, the process returns to the step S302.

  In step S310, the operation mode is set to the still image shooting mode, and in step S311, a normal aperture image is shot with reference to the first imaging condition. In step S312, a small aperture image is captured with reference to the second imaging condition, and in step S313, a defocused image is captured with reference to the third imaging condition. In step S314, the normal aperture image, the small aperture image, and the defocused image thus obtained are stored in the MPF file, and the MPF file file is recorded in the external memory 22. When the process of step S314 is completed, the process returns to step S301.

  Referring to FIG. 30, in step S401, an MPF file recorded in external memory 22 is designated. In step S402, the normal aperture image, small aperture image, and defocus image are reproduced from the designated MPF file. The normal aperture image and the small aperture image are subjected to alignment processing by the alignment unit 104 in step S403.

  In step S404, the high frequency component of the normal aperture image output from the alignment unit 104 is extracted by the high frequency component extraction unit 105, and the high frequency component of the small aperture image output from the alignment unit 104 is extracted. Extracted by the unit 106. The difference between the extracted high frequency components is calculated by the difference calculation unit 107. In step S405, the addition ratio calculation unit 109 executes a calculation process of the addition ratio K with reference to the calculated difference. Thereby, an addition ratio table is created.

  On the other hand, in step S406, alignment processing by the alignment unit 104 is performed on the reproduced normal aperture image and defocused image. In step S <b> 407, the normal aperture image and the defocus image output from the alignment unit 104 are combined by the image combining unit 110. At this time, the addition ratio table created in step S405 is referred to. The composite image output from the image composition unit 110 is output from the display unit 13 in step S408.

1 is an overall configuration diagram of an imaging apparatus according to an embodiment of the present invention. It is an internal block diagram of the lens part 3 of FIG. (A) is a figure for demonstrating the captured image imaged with the imaging device 1 of FIG. 1, (B) is a figure for demonstrating the other captured image imaged with the imaging device 1 of FIG. is there. It is a figure for demonstrating the internal structure of the image process part 7 of FIG. 1, and a part of image processing. 4A is an image diagram of an image after blurring processing by the image processing unit 108 in FIG. 4, and FIG. 4B is an image diagram of an output image from the image synthesis unit 110. It is a graph which shows the function for the addition ratio calculation by the addition ratio calculation part of FIG. FIG. 5 is a diagram illustrating an internal configuration of an image registration processing unit 104 in FIG. 4. It is a figure for demonstrating the position shift (motion vector) detected by the position shift detection part 202 of FIG. It is a figure for demonstrating the position shift correction by the position shift correction part 203 of FIG. 2 is a flowchart showing a part of a procedure for generating a specific subject emphasized image in the imaging apparatus of FIG. 1. 12 is a flowchart showing another part of the procedure for generating the specific subject emphasized image in the imaging apparatus of FIG. 1. It is a flowchart which shows the procedure which produces | generates the specific object emphasis image in step S113 of FIG. It is a figure for demonstrating the internal structure and image processing of the image process part 7 of FIG. 1 in 2nd Embodiment. FIG. 14A is an image diagram of an image after luminance reduction processing by the image processing unit 113 in FIG. 13, and FIG. 14B is an image diagram of an output image from the image composition unit 110. It is a figure for demonstrating the internal structure and image processing of the image process part 7 of FIG. 1 in 3rd Embodiment. It is a figure which shows the color correlation for demonstrating saturation. (A) is an image diagram of an image after saturation reduction processing by the image processing unit 116 in FIG. 15, and (B) is an image diagram of an output image from the image synthesis unit 110. It is a figure which shows the modification of the image process part 7 in 2nd Embodiment. FIG. 19A is an image diagram of an image after the luminance enhancement processing by the image processing unit 119 in FIG. 18, and FIG. 19B is an image diagram of an output image from the image composition unit 110. It is a figure which shows the modification of the image process part 7 in 3rd Embodiment. 20A is an image diagram of an image after saturation enhancement processing by the image processing unit 122 in FIG. 20, and FIG. 20B is an image diagram of an output image from the image synthesis unit 110. It is a part of modification of the flowchart which shows the procedure which produces | generates a specific subject emphasis image. 10 is another part of a modified example of the flowchart showing the procedure for generating the specific subject emphasized image. It is a figure which shows the modification of the image process part. It is a figure which shows the other modification of the image process part. It is a graph which shows the function for the cutoff frequency calculation by the cutoff frequency calculation part of FIG. It is a figure for demonstrating the internal structure and image processing of the image process part 7 of FIG. 1 in 4th Embodiment. 14 is a flowchart illustrating a part of a procedure for generating a specific subject emphasized image in the fourth embodiment. 16 is a flowchart illustrating another part of the procedure for generating the specific subject emphasized image in the fourth embodiment. 15 is a flowchart illustrating another part of a procedure for generating a specific subject emphasized image in the fourth embodiment.

DESCRIPTION OF SYMBOLS 1 ... Imaging device 6 ... Image processing part 100 ... 1st image 103 ... 2nd image 104 ... Image registration part 105 ... High frequency component extraction part 106 ... High frequency component extraction part 107 ... Difference extraction part 109 ... Addition ratio calculation part 108 ... Image processing unit 110 ... Image composition unit 111 ... Third image 112 ... Fourth image

Claims (11)

An image processing device for processing a scene image output from an imaging means that captures a scene through a focus lens and a diaphragm mechanism,
A focus adjusting means for adjusting the focus so as to be adapted to a specific object in the object scene;
A diaphragm amount setting means for setting a plurality of different diaphragm amounts in the diaphragm mechanism in relation to the adjustment process of the focus adjusting means;
Detecting means for detecting a difference in sharpness between a plurality of field images output from the imaging means corresponding to a plurality of aperture amounts set by the aperture amount setting means; and the focus adjustment An image processing apparatus comprising: an image quality adjustment unit that performs an image quality adjustment process with reference to a detection result of the detection unit on the object scene image output from the imaging unit corresponding to the focus adjusted by the unit.   The detection means is a high-frequency component extraction means for extracting a plurality of high-frequency components from the plurality of object scene images, and a difference calculation means for calculating a difference between the plurality of high-frequency components extracted by the high-frequency component extraction means. The image processing apparatus according to claim 1, comprising:   The detection means includes a luminance component extraction means for extracting a plurality of luminance components from the plurality of scene images, and the plurality of scene images based on the plurality of luminance components extracted by the luminance component extraction means. The image processing apparatus according to claim 2, further comprising correction means for correcting a positional deviation between the two.   4. The image processing apparatus according to claim 1, wherein the image quality adjustment unit includes a specific parameter control unit that greatly reduces a specific parameter of the object scene image as the difference increases. 5. The image quality adjusting means further includes a reducing means for reducing a specific parameter of the object scene image,
The image processing apparatus according to claim 4, wherein the specific parameter control unit increases a reduction amount of the reduction unit as the difference increases. First recording means for recording a plurality of object scene images output from the imaging means corresponding to a plurality of aperture amounts set by the aperture amount setting means, and a plurality of objects recorded by the recording means A first reproducing means for reproducing the field image;
The detecting means pays attention to a plurality of scene images reproduced by the first reproducing means,
The image processing apparatus according to claim 1, wherein the image quality adjustment unit focuses on one of a plurality of object scene images reproduced by the first reproduction unit.   The image processing apparatus according to claim 6, wherein the first recording unit stores the plurality of scene images in a common file.   The image quality adjusting means includes a defocus setting means for setting the focus to a defocus state, a scene image output from the imaging means corresponding to the defocus set by the defocus setting means, and the focus adjustment. 5. The image processing apparatus according to claim 1, further comprising a synthesizing unit that synthesizes the object scene image output from the imaging unit corresponding to the focus adjusted by the unit. A plurality of object scene images output from the imaging unit corresponding to a plurality of aperture amounts set by the aperture amount setting unit and a defocus set by the defocus setting unit from the imaging unit. A second recording means for recording the output object scene image; and a second reproducing means for reproducing a plurality of object scene images recorded by the recording means;
The image processing apparatus according to claim 8, wherein the detection unit and the image quality adjustment unit pay attention to a plurality of scene images reproduced by the second reproduction unit.   The image processing apparatus according to claim 9, wherein the second recording unit stores the plurality of scene images in a common file.   An imaging device comprising the image processing device according to claim 1.