JP2009218709A - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and an image processing method that obtain a blur image of higher quality without forcing a user to make complicated settings nor to perform complicated operations. <P>SOLUTION: The image processing apparatus which includes a blurring process means of performing a blurring process on input background image data, and outputs a blurred background image includes a photographic mode decision means of deciding a photographic mode representing a photographic scene when the background image is photographed, and a blur intensity determination means of determining blur intensity of the blurring process based upon the decided photographic mode. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus capable of obtaining a high-quality blurred image.
The present invention is applicable to a digital camera and a digital camera control method.

An image pickup device used in a compact digital camera is smaller than an image pickup device used in a single-lens reflex camera or a silver salt film. Therefore, the focal length of a shooting optical system necessary for shooting an image with the same angle of view is short.
When the focal length is short, the depth of field becomes deep even if the F number of the photographing optical system is the same.
If the F-number can be reduced in proportion to the reduction in the focal length, the depth of field can be reduced, but it is necessary to increase the diameter of the photographing optical system, which increases the size and cost.

For this reason, when a picture is taken with a compact digital camera, a relatively wide distance range is in focus.
This is also an advantage in that an image with less blur can be obtained when shooting images of the same brightness, but when shooting images where background blur is important, such as portraits, even the background There is a problem that it becomes clear and it becomes a disadvantage.

In order to solve such a problem, a camera that blurs the background by image processing has been proposed (see, for example, Patent Documents 1 to 3).
In the cameras described in Patent Documents 1 to 3, the blur intensity is changed according to the distance to the subject, and the blur with a sense of perspective is reproduced.
Further, in Patent Document 3, the blur intensity is converted to be equivalent to that taken in the 135 format. In Patent Document 3, the user can adjust the blur intensity.

  However, for example, if an image photographed at a wide angle is strongly blurred, there is a problem that an unnatural image that cannot be photographed with a lens that exists in the world is generated.

JP 9-318870 A JP-A-11-266388 JP 2003-37767 A

  The present invention has been made in view of the situation as described above, and provides an image processing apparatus and an image processing method capable of preventing a troubled image having a sense of incongruity from being generated by a blurring process. It is an object.

In order to solve the above problems, an image processing apparatus and an image processing method according to the present invention specifically have technical features described in the following (1) to (3).
(1): A blur intensity adjustment for adjusting a blur intensity of the blur processing in an image processing apparatus that includes a blur processing unit that performs a blur process of background image data input via an optical system and outputs a blurred background image. An adjustment range for setting an adjustment range of the blur intensity adjustment unit based on the determined focal length, a focal length determination unit that determines a focal length of the optical system when the background image is captured An image processing apparatus comprising: setting means.

  According to the configuration described in (1) above, since it is configured to set the adjustment range of the blur intensity based on the focal length, it is possible to generate a defective image with a sense of incongruity due to the blur processing. There is an effect that it can be prevented.

(2): In an image processing method for outputting a blurred background image, including a blur processing step for blurring background image data input via an optical system, the blur strength of the blur processing is set prior to the blur processing step. An adjustment range of the blur intensity adjustment step based on the determined focal length, a blur distance adjustment step of adjusting, a focal length determination step of determining a focal length of the optical system when the background image is captured And an adjustment range setting step for setting the image processing method.

  According to the configuration described in (2) above, since it is configured to set the adjustment range of the blur intensity based on the focal length, it is possible to generate a defective image with a sense of incongruity due to the blur processing. There is an effect that it can be prevented.

(3) The image processing apparatus according to (1), wherein the image processing apparatus is a digital camera.

  According to the configuration described in (3) above, since the configuration described in (1) above is configured to be applied to a digital still camera, the same effect as the configuration in (1) above can be obtained.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method that can prevent a defective image having a sense of incongruity from being generated by blurring processing.

  An imaging apparatus according to the present invention includes a blur processing unit that performs blur processing on background image data input via an optical system, and adjusts the blur strength of the blur processing in an image processing device that outputs a blurred background image. Based on the determined focal length, an adjustment range of the blur intensity adjusting unit is set based on a blur intensity adjusting unit, a focal length determining unit that determines a focal length of the optical system when the background image is captured Adjusting range setting means.

The basic configuration of the image processing apparatus of the present invention will be described in detail with reference to the drawings.
The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention is particularly limited in the following description. Unless otherwise described, the present invention is not limited to these embodiments.

FIG. 1 is a schematic block diagram showing the configuration of a digital still camera and its connected devices as an embodiment of an image processing apparatus according to the present invention.
In FIG. 1, 01 is a digital still camera device, 02 is a system control unit, which includes a CPU, a NAND flash memory, an SDRAM, a timer, and the like provided for controlling the entire digital still camera device 01. The imaging unit, 04, which includes an optical system component (lens and lens driving motor), a CCD, a CCD driving circuit, an A / D converter, and the like provided for imaging, Performs various image processing and controls the CCD drive timing and lens drive motor of the imaging unit 03 to perform zooming, focusing, exposure adjustment, etc., and also for image processing provided to compress and expand images An image processing unit 05 including a DSP (digital signal processor), a RAM, etc., displays an image signal processed by the image processing unit on the LCD Display control unit comprising a D / A converter, an on-screen display controller and the like provided for performing signal processing for display and generating various graphic images for a user interface and displaying them on an LCD , 06 is an LCD provided for displaying an image and displaying a graphic for a user interface, and 07 is a memory card controller provided for interfacing with a recording medium. The media interface unit 08 is a recording medium that is detachable from the digital still camera device 01, which includes a flash memory or the like provided for storing various information related to the compressed image signal and the image. User interfaces such as keys and dials not shown A hard key interface unit 10 composed of a sub CPU, etc., provided for detecting the state of the voice and controlling the main power supply to the main CPU, is provided for performing data communication by connecting a USB. The communication interface unit 11 including a communication controller connects the digital still camera device 01 via USB, transfers images from the digital still camera device 01 for playback, and makes various settings on the digital still camera device 01. A PC for performing is shown.

First, the starting operation will be described.
When the user presses a power button (not shown), the hard key interface unit 09 turns on the power supply to the main CPU.
The main CPU in the system control unit 02 first starts access (program execution) from the boot unit of the NAND flash memory, and transfers program data to the SDRAM by the boot program.
When the transfer to the SDRAM is completed, the program execution pointer (program counter) is moved to the transferred program on the SDRAM, and thereafter, the startup process is started by the program on the SDRAM.

The startup processing includes OS (operating system) initialization, lens barrel extension processing, recording media initialization processing, and the like.
The lens barrel feeding process is performed by applying a pulse signal to the lens driving motor of the imaging unit 03 via the image processing unit 04 at predetermined intervals (2 mS).
Also, in the initialization process of the recording medium, after supplying power and a clock to the recording medium 08 via the recording medium interface unit 07, an initialization command is issued to the recording medium 08. The actual initialization process is performed in the recording medium 08, and the system control unit 02 polls the status of the recording medium 08 at intervals of 10 mS in order to detect the completion.

Next, the operation during shooting will be described.
Prior to shooting, the user operates various keys and dials (not shown) to determine the shooting mode (high image quality mode, low image quality mode, etc.).

The user's operation content is determined by the system control unit 02 through the hard key interface unit 09, and the system control unit 02 generates a guidance graphic to the display control unit 05 according to the operation, and prompts the user to perform the next operation.
When the photographing mode is determined, the system control unit 02 sets a processing parameter corresponding to the mode in the image processing unit 04.
Alternatively, the user operates a zoom lever (not shown) to determine the angle of view (composition).

The user's operation content is determined by the system control unit 02 through the hard key interface unit 09, and the system control unit 02 controls the imaging unit 03 in accordance with the operation to drive the lens.
In accordance with control from the image processing unit 04, the imaging unit 03 starts an imaging operation for displaying a monitoring image prior to actual shooting.
The imaged data is continuously sent to the image processing unit 04. The image processing unit 04 performs processing such as color space conversion, gamma correction, and white balance adjustment, and then sends the image data to the display control unit 05.
In the display control unit 05, the image data is signal-processed and displayed on the LCD 06, and the imaging state is presented to the user.

When a release button (not shown) is pressed, the operation is discriminated by the system control unit 02 through the hard key interface unit 09 as in the mode setting.
The imaging unit 03 performs focus adjustment according to the control from the image processing unit 04 and then sends the captured image to the image processing unit 04. The image processing unit 04 performs image processing and compression processing according to the shooting mode.
The system control unit 02 reads the compressed image data, adds header information, and writes the image data to the recording medium through the recording medium interface unit 07.
This completes a series of shooting operations.

[First embodiment (background blur)]
Next, a description will be given of a first embodiment of a method for performing background blurring, which is a feature of the image processing apparatus according to the present invention.
FIG. 2A is a flowchart showing a first embodiment of a background blur operation flow in the digital camera according to the present invention.
The operation flow in the present embodiment shows an operation when the above-described release button is pressed in a state where the blurring processing function is set to ON.

  When the release button is pressed, the system control unit 02 controls the image processing unit 04 and the imaging unit 03 to perform a CCDAF scanning operation (step 01-001).

Here, an outline of CCDAF will be described.
In general, in an electronic image pickup apparatus having a two-dimensional image pickup device such as a digital camera or a video camera, a sharpness of a screen is detected by a video signal photoelectrically converted by the image pickup device, and a focusing lens is set so that the sharpness is maximized. The focus is adjusted by controlling the position of. Normally, the sharpness is small when the focus is blurred, increases as the focus is achieved, and reaches a maximum value when the focus is completely achieved. The CCDAF gradually moves the focus lens from the infinite end to the near end, images the subject at a plurality of positions, and the position of the focus lens from which the image with the highest sharpness is obtained among the plurality of imaged images. Is a method of setting the in-focus position.

  The imaging unit 03 and the image processing unit 04 perform focusing and image capture in the above-described procedure (step 01-002).

  Subsequently, the system control unit 02 calculates the distance for each position of each image (step 01-003).

Hereinafter, a description will be given with reference to FIG.
In FIG. 3A, reference numeral 100 denotes a captured area of the captured image, and 101 denotes one AF evaluation value area.
As shown in the figure, the AF evaluation value area is a small area obtained by equally dividing the imaging area, and an AF evaluation value for each area (integrated value of the contrast of the image in the area) is obtained by scanning with the CCDAF. It is done.
The system control unit 02 analyzes the AF evaluation value for each lens position obtained by CCDAF scanning for each region based on a predetermined algorithm, and determines the lens driving position corresponding to the peak position of the AF evaluation value. To do.
Further, the system control unit 02 converts the lens driving position into distance information from the current zoom position (lens focal length) for each region.

Here, the AF evaluation value is obtained by performing a horizontal HPF (high pass filter) operation on each pixel in the AF evaluation value area and adding the obtained high frequency components.
For example, a value such as ki = {− 1, −2, 6, 6, −2, −1} is used as the HPF coefficient.
k0 is a coefficient to be multiplied by the pixel of the pixel in the horizontal direction-2 of the target pixel, k1 is a coefficient to be multiplied to the pixel of the pixel of interest in the horizontal direction-1 coordinate, k2 is a coefficient to be multiplied by the target pixel, and k3 is A coefficient to be multiplied by a pixel having a coordinate in the horizontal direction + 1 of the target pixel, and k4 is a coefficient to be multiplied by a pixel having a coordinate in the horizontal direction + 2 of the target pixel.

  In order to obtain the distance information from the AF evaluation value, it is obtained by the following equation based on the Gaussian imaging equation.

1 / a + 1 / b = 1 / f
a = b × f / (b−f)
(Where, a is the distance from the lens to the subject, b is the distance between the lens and the image sensor, and f is the focal length of the lens.)

Here, the distance a from the lens to the subject is the distance information to be obtained. The focal length f of the lens is uniquely obtained from the zoom position at the time of photographing, and the distance b between the lens and the image sensor is uniquely obtained from the driving position of the focus lens from which the peak of the AF evaluation value is obtained.
As described above, it is possible to obtain distance information for each AF evaluation value area 101 of the entire region within the angle of view 100.

In FIG. 3A, reference numeral 102 denotes an area focused by AF.
The system control unit 02 determines the area closest to the AF evaluation value area at the center of the screen as the AF area, and further determines the block equidistant from the AF area as the main subject block.

In FIG. 3B, reference numeral 103 denotes a main subject block (the main subject block 103 includes the AF area 102).
The system control unit 02 determines the main subject area based on the obtained information on the main subject block 103 and the captured image (step 01-004). In this process, a region having an arbitrary shape including the main subject block 103 is determined by conventional image processing (contour extraction).

In FIG. 3C, reference numeral 104 denotes a main subject area.
The image processing unit 04 sequentially performs main subject image extraction processing, background image blurring processing, and synthesis processing based on the information of the main subject region 104 (step 01-005; blur processing means).

In FIG. 3D, 105 is a captured image, 106 is a main subject, 107 is an extracted main subject image, 108 is a background image, 109 is a blurred background image, and 110 is a composite image.
In the main subject extraction process, the main subject is extracted by separating the image along the main subject region 104. As a result, the captured image 105 is separated into the main subject image 107 and the background image 108.

  In the background image blurring process, a spatial filter is applied to the background image 108 to generate a blurred background image 109.

  The spatial filter calculates a smoothing filter (k (ix, iy)) on the input image (In (x, y)) as shown in Equation 1 below, and obtains an output image (Out (x, y)).

X and y in the input image (In (x, y)) indicate coordinate values (horizontal coordinate x, vertical coordinate y) of the target pixel. In Equation 1, the coordinate calculation results (x + ix−fs / 2 and y + iy−fs / 2) are clipped so as to be rounded down to an integer and point to the input image.
Ix and iy in the filter coefficient (k (ix, iy)) indicate the position of the filter coefficient (horizontal direction ix, vertical direction iy).

According to the above formula 1, the value of the target pixel is replaced with the average value of the surrounding pixels (one side is a square having a size fs).
When fs increases, the size of the area to be averaged increases, and a strong blurring effect is obtained.
In the case of fs = 1, since the average value of only the target pixel is obtained, the value does not change and there is no blurring effect.

  In the present embodiment, the value of fs is processed with the initial value 3 every time the image is taken, and is updated by changing the blur intensity (blur intensity adjusting means, focal length determining means, adjustment range setting means) described later.

In the synthesizing process, the main subject image 107 is superimposed on the blurred background image 109 and synthesized to generate a synthesized image 110.
The generated composite image is displayed on the LCD 06 via the display control unit 05 (step 01-006).

Here, when an operation for changing the blur intensity is performed (step 01-007), the system control unit 02 subtracts or adds 2 to the value of fs by 2 according to the operation (step 01-009) and performs image processing again. However, prior to the change, the focal length f of the lens is read, and based on the focal length f of the lens, it is determined whether or not the value of fs after the change is equal to or less than a predetermined limit value (adjustment range) (step 01-008). If the limit value is exceeded, no change is made.
Also, no change is made when the value of fs is less than 1 (adjustment range).
In the present embodiment, the value of fs (blurring intensity) is subtracted or added by 2 according to the operation. However, the present invention is not limited to this, and any value is subtracted or added. can do.
An example of the relationship between the focal length f of the lens and the limit value of the predetermined fs is shown in Table 1 below.

  When the confirmation operation is performed (step 01-010), the system control unit 02 controls the image processing unit 04 to compress the combined image, reads the compressed image data, and adds header information. Then, 08 is written to the recording medium through the recording medium interface unit 07 (step 01-011). At this time, the obtained distance information is added to the header information.

[Second embodiment (background blur)]
Next, a second embodiment of a method for performing background blurring, which is a feature of the image processing apparatus according to the present invention, will be described. The present embodiment is an example in which blurring processing is performed on a background portion other than the main subject during reproduction.
FIG. 2B is a flowchart showing a second embodiment of the background blur operation flow in the digital camera according to the present invention.
The operation flow in the present embodiment shows the operation when the blurring process is selected by the user's operation during image reproduction.

  The system control unit 02 reads the assigned distance information from the header information of the image file being reproduced (step 02-001).

  The main subject area determination, main subject image extraction processing, background image blurring processing, composition processing, and display are the same as in steps 01-004 to 006 in the first embodiment (steps 02-002 to 004).

When the operation for changing the blur intensity is performed (step 02-005), the system control unit 02 reads the attached focal length information from the header information of the image file being reproduced (step 02-006), and the first control is performed. The limit value is determined in the same manner as in step 01-008 in the embodiment (step 02-007). Steps 02-005 to 007 correspond to the blur intensity adjusting unit, the focal length determining unit, and the adjustment range setting unit in the present embodiment.
The subsequent operations (steps 02-008 to 010) are the same as steps 01 to 009 to 011 in the first embodiment.

  As described above, according to the first and second embodiments, it is possible to prevent the generation of a defective image having an uncomfortable feeling due to the blurring process.

1 is a schematic block diagram illustrating a configuration of a digital still camera and an apparatus connected thereto as an embodiment of an image processing apparatus according to the present invention. (A) It is 1st Embodiment of a background image blurring process. (B) It is 2nd Embodiment of a background image blurring process. It is a schematic diagram which shows the imaging flow in the image processing apparatus which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital still camera apparatus 2 System control part 3 Imaging part 4 Image processing part 5 Display control part 6 LCD
7 Recording media interface unit 8 Recording media 9 Hard key interface unit 10 Communication interface unit 11 PC

Claims (3)

In an image processing apparatus that includes a blur processing unit that performs blur processing on background image data input via an optical system, and outputs a blurred background image.
Blur intensity adjusting means for adjusting the blur intensity of the blur processing;
A focal length determination means for determining a focal length of the optical system when the background image is captured;
An image processing apparatus comprising: an adjustment range setting unit that sets an adjustment range of the blur intensity adjustment unit based on the determined focal length. In an image processing method that includes a blurring process step of blurring background image data input via an optical system and outputs a blurred background image,
A blur intensity adjusting step for adjusting the blur strength of the blur processing prior to the blur processing step;
A focal length determination step of determining a focal length of the optical system when the background image is captured;
And an adjustment range setting step of setting an adjustment range of the blur intensity adjustment step based on the determined focal length.   The image processing apparatus according to claim 1, wherein the image processing apparatus is a digital camera.