JP2009218704A - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and an image processing method that obtain a blur of high picture quality equivalent to that of optical blurring. <P>SOLUTION: In the image processing apparatus which includes a blurring process means of performing a blurring process on input background image data and outputs the blurred background image, the blurring process means includes a conversion means of amplifying background image data of each pixel when the value of the background image data of the each pixel is high and an inverse conversion means of performing conversion inverse to that of the conversion means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus capable of obtaining high-quality blur.
The present invention is applicable to a digital camera and a digital camera control method.

An image pickup device used in a compact digital camera is smaller than an image pickup device used in a single-lens reflex camera or a silver salt film. Therefore, the focal length of a shooting optical system necessary for shooting an image with the same angle of view is short.
When the focal length is short, the depth of field becomes deep even if the F number of the photographing optical system is the same.
If the F-number can be reduced in proportion to the reduction in the focal length, the depth of field can be reduced, but it is necessary to increase the diameter of the photographing optical system, which increases the size and cost.

For this reason, when a picture is taken with a compact digital camera, a relatively wide distance range is in focus.
This is an advantage in that an image with less blur can be obtained when shooting images of the same brightness, but when shooting images where background blur is important, such as portraits, even the background There is a problem that it becomes clear and it becomes a disadvantage.

In order to solve this problem, cameras that blur the background by image processing have been proposed (Patent Documents 1 to 4).
In the cameras described in Patent Documents 1 to 4, the characteristics of the filter are changed according to the distance and the position within the angle of view, and blur with a sense of perspective is reproduced.

  However, when the blurring process is performed only by the image processing using the filter as described above, unlike the case of optically blurring, the blurring of the high-luminance portion becomes a blurring method with low contrast.

This will be described with reference to FIG.
FIG. 1 is a diagram illustrating an example of conventional image processing in which blur processing is performed on a point image with high luminance and an example of image processing in the present invention.
When a point image with high brightness as shown at 501 is taken, an optical image is formed on the CCD with a waveform as shown at 502. Reference numeral 502 denotes a waveform indicating the intensity of light for one line crossing the point image.

When the image is actually optically blurred, the high luminance level is dispersed, the light hitting the CCD changes to the intensity as shown at 503, and the image data taken similarly shows the waveform as shown at 504. As a result, a blurred image such as 505 is formed.
On the other hand, when the image is captured in focus, the captured image data is clipped at the maximum luminance level (255) and shows a waveform as indicated by 506.
If the blurring process is performed on the waveform as shown in 506 as it is, the maximum brightness level is dispersed and the brightness level is lowered as shown in 507, resulting in a blurred image with a low contrast like 508.

JP-A-11-266388 JP 2003-37767 A JP 2003-101858 A JP 9-318870 A

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus and an image processing method capable of obtaining a high-quality blur equivalent to that obtained when optically blurred. .

  In order to solve the above problems, an image processing apparatus and an image processing method according to the present invention specifically have technical features described in the following (1) to (17).

(1): In an image processing apparatus that includes a blur processing unit that performs blur processing of input background image data, and outputs a blurred background image, the blur processing unit has a high value of the background image data in each pixel In this case, the image processing apparatus includes a conversion unit that amplifies the background image data in the pixel and an inverse conversion unit that performs an inverse conversion of the conversion unit.

  According to the configuration of (1), since it is configured to amplify image data with a high level value based on the color level value of each pixel of the image data in the process of blurring, it is optically blurred. There is an effect that a high-quality blur equivalent to the case can be obtained.

(2): In an image processing apparatus that includes a blur processing unit that performs blur processing of input background image data and outputs a blurred background image, the blur processing unit has a high luminance value in each pixel. An image processing apparatus comprising: conversion means for amplifying background image data in the pixel; and inverse conversion means for performing inverse conversion of the conversion means.

  According to the configuration of (2), in the process of blurring, it is configured to amplify image data with a high level value based on the luminance level value of each pixel of the image data. There is an effect that a high-quality blur equivalent to the above can be obtained.

(3): In the image processing apparatus according to the above (1) or (2), after the conversion by the conversion unit, the filter process is performed, and after the filter process is performed, the inverse conversion by the inverse conversion unit is performed. An image processing apparatus characterized by the above.

  According to the configuration of (3), the filter processing is performed after the value of each pixel of the image data is amplified, and the inverse conversion is performed after the filter processing is performed. There is an effect that an equivalent high-quality blur can be obtained.

(4): In the image processing device according to any one of (1) to (3), the conversion unit further includes an image data amount expansion function for performing conversion for expanding an image data amount per pixel. An image processing apparatus comprising:

  According to the configuration of (4), since it is configured to further increase and enlarge the amount of image data per pixel by conversion, it is possible to prevent deterioration of data accuracy associated with conversion, and consequently high quality. There is an effect that blurring processing can be performed.

(5): The image processing apparatus according to any one of (1) to (3), further including a setting unit that sets an amplification degree of the conversion unit.

  According to the configuration of (5), since it is configured so that the degree of amplification by conversion can be set, there is an effect that the image quality of blur can be optimally controlled.

(6): The image processing apparatus according to (5), wherein the setting unit sets an amplification degree based on a shooting mode.

  According to the configuration of (6), since the amplification degree by the conversion is configured to be set based on the shooting mode, there is an effect that it is possible to obtain an optimal image quality blur according to the shooting mode.

(7): In the image processing device according to (5), the setting means sets an amplification degree based on characteristics of background image data.

According to the configuration of (7), since the degree of amplification by conversion is configured to be set based on the characteristics of the entire image data, there is an effect that it is possible to obtain an optimum image quality blur according to the image. is there.

(8): In the image processing apparatus according to (7), the setting unit sets the amplification degree based on an average level of luminance values and a maximum level of luminance values among characteristics of background image data. An image processing apparatus characterized by this.

  According to the configuration of (8), the amplification degree by the conversion is configured to be set based on the average level of the luminance value and the maximum level of the luminance value among the characteristics of the image data. There is an effect that it is possible to obtain an optimum image quality blur according to the scene.

(9): In an image processing method that includes a blurring process step that blurs input background image data and outputs a blurred background image, the blurring process step has a high value of the background image data in each pixel In this case, the image processing method includes a conversion step for amplifying the background image data in the pixel and an inverse conversion step for performing the inverse conversion of the conversion means.

  According to the configuration of (9), since it is configured to amplify image data having a high level value based on the color level value of each pixel of the image data in the process of blurring, it is optically blurred. There is an effect that a high-quality blur equivalent to the case can be obtained.

(10): In an image processing method that includes a blurring process step for blurring input background image data and outputs a blurred background image, the blurring process step is performed when the luminance value in each pixel is high. An image processing method comprising: a conversion step for amplifying background image data in the pixel; and an inverse conversion step for performing an inverse conversion of the conversion means.

  According to the configuration of (10), in the process of blurring, it is configured to amplify image data with a high level value based on the luminance level value of each pixel of the image data. There is an effect that a high-quality blur equivalent to the above can be obtained.

(11): The image processing method according to (9) or (10), wherein after the conversion step, filter processing is performed, and after the filter processing is performed, the inverse conversion step is performed. It is a processing method.

  According to the configuration of (11), the filter processing is performed after the value of each pixel of the image data is amplified, and the inverse conversion is performed after the filter processing is performed. There is an effect that an equivalent high-quality blur can be obtained.

(12) In the image processing method according to any one of (9) to (11), the conversion step further includes an image data amount expansion function for performing conversion for expanding an image data amount per pixel. It is an image processing method characterized by having.

  According to the configuration of (12), the image data amount per pixel is further increased and expanded by the conversion, so that it is possible to prevent the deterioration of the data accuracy associated with the conversion, and as a result, high quality. There is an effect that blurring processing can be performed.

(13): In the image processing method described in any one of (9) to (11) above, the amplification degree of the conversion step is variable by setting.

  According to the configuration of (13), since it is configured so that the degree of amplification by conversion can be set, there is an effect that the image quality of blur can be optimally controlled.

(14): The image processing method according to (13), wherein the setting is based on a shooting mode.

  According to the configuration of (14), since the amplification degree due to the conversion is configured to be set based on the shooting mode, there is an effect that it is possible to obtain an optimal image quality blur according to the shooting mode.

(15): In the image processing method according to (13), the setting is based on characteristics of background image data.

According to the configuration of (15), the degree of amplification by conversion is configured to be set based on the characteristics of the entire image data, so that an effect of obtaining an optimum image quality blur according to the image can be obtained. is there.

(16): In the image processing device according to (15), the setting is based on an average level of luminance values and a maximum level of luminance values among characteristics of background image data. It is.

  According to the configuration of (16), the amplification degree by the conversion is configured to be set based on the average level of the luminance value and the maximum level of the luminance value among the characteristics of the image data. There is an effect that it is possible to obtain an optimum image quality blur according to the scene.

(17) The image processing apparatus according to any one of (1) to (8), wherein the image processing apparatus is a digital camera.

  According to the configuration of (17), since the configuration described in any one of (1) to (8) is applied to a digital still camera, high-quality blur is immediately applied to a captured image. It is possible to perform processing, and there is an effect that there is a great effect as compared with other devices.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method capable of obtaining high-quality blur equivalent to that obtained when optically blurred.

The image processing apparatus of the present invention includes a blur processing unit that performs blur processing of input background image data, and the blur processing unit outputs the blurred background image. The blur processing unit includes the background image data in each pixel. When the value of is high, it has a conversion means for amplifying background image data in the pixel, and an inverse conversion means for performing an inverse conversion of the conversion means.
The image processing apparatus according to the present invention further includes a blur processing unit that performs blur processing on the input background image data. In the image processing device that outputs a blurred background image, the blur processing unit has a luminance value at each pixel. If it is high, it has a conversion means for amplifying the background image data in the pixel and an inverse conversion means for performing an inverse conversion of the conversion means.

  That is, in the present invention, in FIG. 1, the waveform 506 is once converted into a waveform 509 and then subjected to blurring processing to obtain a waveform 510, and optically blurred like a waveform 511 after inverse conversion. In this case, a waveform close to the image data 504 is obtained.

Next, the basic configuration of the image processing apparatus of the present invention will be described in detail with reference to the drawings.
The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention is particularly limited in the following description. Unless otherwise described, the present invention is not limited to these embodiments.

FIG. 2 is a schematic block diagram showing the configuration of a digital still camera and its connected devices as an embodiment of an image processing apparatus according to the present invention.
In FIG. 2, 01 is a digital still camera device, 02 is a system control unit, which includes a CPU, a NAND flash memory, an SDRAM, a timer, etc., provided to control the entire digital still camera device 01. The imaging unit, 04, which includes an optical system component (lens and lens driving motor), a CCD, a CCD driving circuit, an A / D converter, and the like provided for imaging, is an image signal obtained by the imaging unit. Performs various image processing and controls the CCD drive timing and lens drive motor of the imaging unit 03 to perform zooming, focusing, exposure adjustment, etc., and also for image processing provided to compress and expand images An image processing unit 05 including a DSP (digital signal processor), a RAM, etc., displays an image signal processed by the image processing unit on the LCD Display control unit comprising a D / A converter, an on-screen display controller and the like provided for performing signal processing for display and generating various graphic images for a user interface and displaying them on an LCD , 06 is an LCD provided for displaying an image and displaying a graphic for a user interface, and 07 is a memory card controller provided for interfacing with a recording medium. The media interface unit 08 is a recording medium that is detachable from the digital still camera device 01, which includes a flash memory or the like provided for storing various information related to the compressed image signal and the image. User interfaces such as keys and dials not shown A hard key interface unit 10 composed of a sub CPU, etc., provided for detecting the state of the voice and controlling the main power supply to the main CPU, is provided for performing data communication by connecting a USB. The communication interface unit 11 including a communication controller connects the digital still camera device 01 via USB, transfers images from the digital still camera device 01 for playback, and makes various settings on the digital still camera device 01. A PC for performing is shown.

First, the starting operation will be described.
When the user presses a power button (not shown), the hard key interface unit 09 turns on the power supply to the main CPU.
The main CPU in the system control unit 02 first starts access (program execution) from the boot unit of the NAND flash memory, and transfers program data to the SDRAM by the boot program.
When the transfer to the SDRAM is completed, the program execution pointer (program counter) is moved to the transferred program on the SDRAM, and thereafter, the startup process is started by the program on the SDRAM.

The startup processing includes OS (operating system) initialization, lens barrel extension processing, recording media initialization processing, and the like.
The lens barrel feeding process is performed by applying a pulse signal to the lens driving motor of the imaging unit 03 via the image processing unit 04 at predetermined intervals (2 mS).
Also, in the initialization process of the recording medium, after supplying power and a clock to the recording medium 08 via the recording medium interface unit 07, an initialization command is issued to the recording medium 08. The actual initialization process is performed in the recording medium 08, and the system control unit 02 polls the status of the recording medium 08 at intervals of 10 mS in order to detect the completion.

Next, the operation during shooting will be described.
Prior to shooting, the user operates various keys and dials (not shown) to determine the shooting mode (high image quality mode, low image quality mode, etc.).

The user's operation content is determined by the system control unit 02 through the hard key interface unit 09, and the system control unit 02 generates a guidance graphic to the display control unit 05 according to the operation, and prompts the user to perform the next operation.
When the photographing mode is determined, the system control unit 02 sets a processing parameter corresponding to the mode in the image processing unit 04.
Alternatively, the user operates a zoom lever (not shown) to determine the angle of view (composition).

The user's operation content is determined by the system control unit 02 through the hard key interface unit 09, and the system control unit 02 controls the imaging unit 03 in accordance with the operation to drive the lens.
In accordance with control from the image processing unit 04, the imaging unit 03 starts an imaging operation for displaying a monitoring image prior to actual shooting.
The imaged data is continuously sent to the image processing unit 04. The image processing unit 04 performs processing such as color space conversion, gamma correction, and white balance adjustment, and then sends the image data to the display control unit 05.
In the display control unit 05, the image data is signal-processed and displayed on the LCD 06, and the imaging state is presented to the user.

When a release button (not shown) is pressed, the operation is discriminated by the system control unit 02 through the hard key interface unit 09 as in the mode setting.
The imaging unit 03 performs focus adjustment according to the control from the image processing unit 04 and then sends the captured image to the image processing unit 04. The image processing unit 04 performs image processing and compression processing according to the shooting mode.
The system control unit 02 reads the compressed image data, adds header information, and writes the image data to the recording medium through the recording medium interface unit 07.
This completes a series of shooting operations.

[First embodiment (background blur)]
Next, a first embodiment of a method for blurring a background portion (background blurring) according to a shooting mode, which is a feature of the image processing apparatus according to the present invention, will be described.
FIG. 3A is a flowchart showing the first embodiment of the background blur operation flow in the digital camera according to the present invention.
The operation flow is shown in FIG. This flow shows the operation related to the operation during monitoring.
When the monitoring operation is started, the system control unit 02 sets a blurring amount parameter, which will be described later, to an initial value (= 5) (step 01-001).
The system control unit 02 controls the image processing unit 04 and the imaging unit 03 to perform a CCDAF scanning operation (step 01-002).
Subsequently, the system control unit 02 determines the distance for each image position (step 01-003).

Hereinafter, a description will be given with reference to FIG.
FIG. 4 is a schematic diagram showing an imaging flow in the image processing apparatus according to the present invention.
In FIG. 4A, 100 indicates the angle of view of the monitoring image, and 101 indicates one AF evaluation value area.
As shown in the figure, the AF evaluation value area is a small area that is uniformly divided within the angle of view, and an AF evaluation value for each area (integrated value of the contrast of the image in the area) is obtained by CCDAF.
The system control unit 02 analyzes the AF evaluation value for each lens position obtained by CCDAF scanning for each region based on a predetermined algorithm, and determines the lens driving position corresponding to the peak position of the AF evaluation value. To do.
Further, the system control unit 02 converts the lens driving position from the current zoom position into distance information for each area.

  Here, a contrast type CCDAF (also referred to as a hill-climbing type) will be described. The contrast type CCDAF first performs imaging while changing the position of the focus lens in minute steps, and extracts a high-frequency portion of each imaging data. Next, the AF evaluation value for each step is calculated based on the extracted high-frequency component, and the position of the focus lens that focuses on the subject is determined based on the change in the AF evaluation value.

Here, the AF evaluation value is obtained by calculating the HPF in the horizontal direction for each pixel in the area and adding the obtained results.
For example, a value such as ki = {− 1, −2, 6, 6, −2, −1} is used as the HPF coefficient.
k0 is a coefficient to be multiplied by the pixel of the pixel in the horizontal direction-2 of the target pixel, k1 is a coefficient to be multiplied to the pixel of the pixel of interest in the horizontal direction-1 coordinate, k2 is a coefficient to be multiplied by the target pixel, and k3 is A coefficient to be multiplied by a pixel having a coordinate in the horizontal direction + 1 of the target pixel, and k4 is a coefficient to be multiplied by a pixel having a coordinate in the horizontal direction + 2 of the target pixel.

  In order to obtain the distance information from the AF evaluation value, it is obtained by the following equation based on the Gaussian imaging equation.

1 / a + 1 / b = 1 / f
a = b × f / (b−f)
(Where, a is the distance from the lens to the subject, b is the distance between the lens and the image sensor, and f is the focal length of the lens.)

Here, the distance a from the lens to the subject is the distance information to be obtained. The focal length f of the lens is uniquely obtained from the zoom position at the time of photographing, and the distance b between the lens and the image sensor is uniquely obtained from the driving position of the focus lens from which the peak of the AF evaluation value is obtained.
As described above, it is possible to obtain distance information for each AF evaluation value area 101 of the entire region within the angle of view 100.

In FIG. 4A, reference numeral 102 denotes an AF area set as an area to be focused by AF.
The system control unit 02 determines the area closest to the AF evaluation value area in the center of the screen as the AF area (step 01-004), and further determines the block equidistant from the AF area as the main subject block. (Step 01-005).

In FIG. 4B, reference numeral 103 denotes a main subject block (the main subject block 103 includes the AF area 102).
At this time, the system control unit 02 calculates and stores the average luminance of the image data at the position corresponding to the main subject block 103 (step 01-006).
Further, the system control unit 02 determines the main subject area based on the obtained information on the main subject block 103 and the captured image (step 01-007). In this process, a region having an arbitrary shape including the main subject block 103 is determined by conventional image processing (contour extraction).

In FIG. 4C, reference numeral 104 denotes a main subject area.
The image processing unit 04 sequentially performs main subject image extraction processing, background image blurring processing, and composition processing based on the information of the main subject region 104 (steps 01-008 to 010).

In FIG. 4D, 105 is a captured image, 106 is a main subject, 107 is an extracted main subject image, 108 is a background image, 109 is a blurred background image, and 110 is a composite image.
In the main subject extraction process (step 01-008), the main subject is extracted by separating the image along the main subject region 104. As a result, the captured image 105 is separated into the main subject image 107 and the background image 108.

  In the background image blurring process (step 01-009), the blurring process based on the blurring amount parameter is performed on the background image 108 to generate a blurred background image 109. Here, the details of the background image blurring process (step 01-009) will be described based on the flow of FIG.

  The system control device 02 determines data conversion parameters for determining an LUT (lookup table) for converting and amplifying the luminance data of each pixel of the background image data 108 based on the current shooting mode. In the present embodiment, since this data conversion is performed by referring to an LUT (lookup table), specifically, which LUT is used for data conversion is selected.

For example, this selection is performed by referring to a table as shown in Table 1, and the contents of the LUT (conversions A to C) used for data conversion are determined according to the shooting mode (step 02-001).
Table 1 is an LUT type determination table based on the shooting mode, where the shooting mode is the current shooting mode (one of the characteristics of the background image data), and the LUT indicates the LUT type used for converting the content data of the table. ing.

FIG. 5 shows a graph of the contents of each table LUT.
In FIG. 5, the horizontal axis indicates the value of the luminance component of the background image data 108, and the vertical axis indicates the value after conversion by the LUT.
Reference numeral 601 indicates the conversion A, 602 indicates the conversion B, and 603 indicates the contents of the conversion C (LUT type, that is, the degree of amplification in the present invention).

By this determination, for example, the night scene mode (conversion C; 603) acts to perform a conversion that emphasizes a higher luminance portion than the normal photographing mode (conversion A; 601).
The content 602 of conversion B acts to amplify the high luminance portion more strongly than the low luminance portion compared to the content 601 of conversion A, and further the content 603 of conversion C is compared to the content 602 of conversion B. Similarly, it acts to amplify more strongly. Therefore, in the night view mode, the higher brightness portion is emphasized than in the normal shooting mode, and it is possible to realize a blur in which the highlight portion peculiar to the night view is greatly spread.
The portrait mode is located between the normal shooting mode and the night view mode, and can emphasize the sunbeams and the like often seen in the background of portrait photographs.

Here, as apparent from the value on the vertical axis, the luminance component of the background image data 108 is represented by 8 bits of data from 0 to 255, whereas the data after LUT conversion is 12 bits of 0 to 4095. Data is output. (Expansion of image data volume)
As a result, it is possible to prevent gradation jumps associated with the conversion and realize high-quality blur.
Note that the conversion shown in FIG. 5 may be obtained by calculation.

The smoothing filter processing described later is performed on the data expanded to 12 bits, and after the smoothing filter processing, the data is returned again to 8-bit data by inverse conversion described later.
Note that the background image data 108 is a YUV format data format represented by luminance and color difference, and this conversion is not performed on the color difference UV, smoothing filter processing is performed with 8 bits remaining, and reverse conversion is also performed. Absent.

The system control unit 02 sets the determined LUT in the image processing unit 04, and the image processing 04 performs conversion by the LUT (spte02-002).
Subsequently, the system control unit 02 sets a smoothing filter of the size specified by the blurring amount parameter (5 × 5 in the case of 5) to the image processing unit 02, and the image processing unit 04 performs the smoothing filter processing (step 02− 003).
In the smoothing filter process, a smoothing filter (k (ix, iy)) is calculated on the input image (In (x, y)) as shown in the following formula 1, and an output image (Out (x, y)) is obtained.

In the present embodiment, k (ix, iy) = 1, ix: 0 to fs-1, and iy: 0 to fs-1. In Equation 1, the coordinate calculation results (x + ix−fs / 2 and y + iy−fs / 2) are clipped so as to be rounded down to an integer and point to the input image.
The blur amount parameter is a parameter that is changed by a user operation, and controls the amount of blur amount. This parameter changes the size of the smoothing filter described above.
In the above equation 1, fs corresponds to the blur parameter, and in step 01-015, which will be described later, the value changes by 1 according to the user's operation.

X and y in the input image (In (x, y)) indicate coordinate values (horizontal coordinate x, vertical coordinate y) of the target pixel.
Ix and iy in the smoothing filter (k (ix, iy)) indicate the positions of the smoothing filter coefficient (horizontal direction ix, vertical direction iy).

According to the above formula 1, the value of the target pixel is replaced with the average value of the surrounding pixels (one side is a square having a size fs).
When fs increases, the size of the area to be averaged increases, and a strong blurring effect is obtained.
In the case of fs = 1, since the average value of only the target pixel is obtained, the value does not change and there is no blurring effect.
The smoothing filter processing is performed on the data after the LUT conversion of the luminance component Y of the background image data 108 and each of the color difference components U and V.

Subsequently, the system control unit 02 sets a table for performing LUT reverse conversion determined in step 03-001 in the image processing unit 04, and the image processing 04 performs conversion by LUT (step 02-004).
By this inverse conversion, the luminance data is once expanded to 12 bits, and the luminance data returns to the original 8-bit data.

Hereinafter, the description will return to the flow of FIG.
In the synthesis process (step 01-010), the main subject image 107 is superimposed on the blurred background image 109 to perform synthesis, and a synthesized image 110 is generated.
The generated composite image is displayed on the LCD 06 via the display control unit 05 (step 01-011).
This completes the processing of one frame of the monitoring image.

At this time, the system control unit 02 calculates the average luminance of the image data at the position corresponding to the main subject block 103 (step 01-012), compares it with the value calculated and stored in step 01-006, and compares the difference by a predetermined amount or more. If there is, the process proceeds to the CCDAF scanning operation again (step 01-013).
Further, when an operation for changing the blur amount is performed, the blur amount parameter is changed according to the operation (step 01-014, 015), and the above operation is repeatedly executed until the monitoring is completed (step 01-016). In this embodiment, in step 01-015, the blurring amount parameter is changed by 1 (by the minimum unit) according to the user's operation.
(The monitoring image processing for each frame is repeated from step 01-007.)

When the release button is pressed, the same blurring process as in steps 01-007 to 010 is performed on the captured image, and an image with a blurred background is recorded.
In this case, (blurring amount parameter during monitoring) × (recording image size) / (monitoring image size) is applied as the blurring amount parameter.

[Second embodiment (background blur)]
Next, as a second embodiment of background blurring in the digital camera according to the present invention, an example of changing the background blurring method based on the average level and the maximum level of image data will be described.
In the second embodiment, the content of the blurring process of the background image in step 01-009 in FIG. 3A is different, and the other flow is the same as in the first embodiment.

The details of the blurring process of the second embodiment will be described based on the flow of FIG.
First, the system control unit 02 calculates the average value and the maximum value of the luminance of the background image 108 (step 03-001).
Then, based on the calculated average value and maximum value, a data conversion parameter for determining an LUT (lookup table) for converting and amplifying the luminance data of each pixel of the background image data 108 is determined. In the present embodiment, this data conversion is performed by referring to an LUT (Look Up Table). Specifically, which LUT is used for data conversion is selected.
For example, this determination and selection are performed by referring to a table as shown in Table 2, and the contents of the LUT (conversions A to C) used for data conversion are determined from the average value and the maximum value of the luminance (step 03-002).
Table 2 is an LUT type determination table based on the average value / maximum value of the luminance. The average value is the average value of the calculated luminance, the maximum value is the maximum value of the calculated luminance, and the LUT is used to convert the content data of the table. LUT type is shown. The contents of each LUT type in the table are the same as those in the first embodiment.

By this determination, for example, for an image including a portion with high luminance on a dark background as a whole, it acts to perform conversion that emphasizes the portion with higher luminance.
As described above, the content 602 of conversion B acts to amplify the high luminance portion more strongly than the low luminance portion compared to the content 601 of conversion A, and further the content 603 of conversion C is the content of conversion B. Compared to 602, it also acts to amplify more strongly. Therefore, when the maximum luminance value is large, that is, when a high-luminance part is included in the image, it is possible to realize a blur in which the highlight part is emphasized and the highlight part is spread. Further, when the average value of the brightness as in the night view is small, the conversion C is selected, and the higher brightness portion is amplified, so that the blur where the highlight portion specific to the night view is greatly spread can be realized.
The subsequent processing is the same as that in the first embodiment.

In the first and second embodiments described above, an example in which the format of the background image data is the YUV format represented by the luminance and the color difference has been shown. However, the present invention uses the intensity of each color as in the RGB format. The present invention can also be applied to image data represented by color signals. In that case, the same effect can be obtained by applying the same processing as the above-described processing for the luminance component to each of the color components (in the case of RGB format, R, G, B).
However, in the RGB format, each color component is processed, whereas in the YUV format, only the luminance component (Y) needs to be processed. Therefore, as in the first and second embodiments. In the case of processing in the YUV format, the processing amount is small and the processing can be performed at high speed.

It is a figure which shows the example of the conventional image processing which adds a blurring process to a high brightness | luminance point image, and the example of the image processing in this invention. 1 is a schematic block diagram illustrating a configuration of a digital still camera and an apparatus connected thereto as an embodiment of an image processing apparatus according to the present invention. (A) It is a flowchart which shows the operation | movement flow of the background blurring in the digital camera which concerns on this invention. (B) It is 1st Embodiment of a background image blurring process. (C) A second embodiment of background image blurring processing. It is a schematic diagram which shows the imaging flow in the image processing apparatus which concerns on this invention. It is the graph which showed the processing result for every LUT classification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital still camera apparatus 2 System control part 3 Imaging part 4 Image processing part 5 Display control part 6 LCD
7 Recording media interface unit 8 Recording media 9 Hard key interface unit 10 Communication interface unit 11 PC

Claims (17)

In an image processing apparatus that includes a blur processing unit that performs blur processing of input background image data, and outputs a blurred background image.
The blur processing means, when the value of the background image data in each pixel is high, conversion means for amplifying the background image data in the pixel;
An image processing apparatus comprising: inverse conversion means for performing inverse conversion of the conversion means. In an image processing apparatus that includes a blur processing unit that performs blur processing of input background image data, and outputs a blurred background image.
The blur processing means, when the luminance value in each pixel is high, conversion means for amplifying the background image data in the pixel;
An image processing apparatus comprising: inverse conversion means for performing inverse conversion of the conversion means. The image processing apparatus according to claim 1 or 2,
After the conversion by the conversion means, filter processing is performed,
An image processing apparatus that performs inverse transformation by the inverse transformation means after performing the filter processing. The image processing apparatus according to any one of claims 1 to 3,
The image processing apparatus according to claim 1, wherein the conversion unit further includes an image data amount expansion function for performing conversion for expanding an image data amount per pixel. The image processing apparatus according to any one of claims 1 to 3,
An image processing apparatus comprising setting means for setting an amplification degree of the conversion means. The image processing apparatus according to claim 5.
The image processing apparatus characterized in that the setting means sets the amplification degree based on a photographing mode. The image processing apparatus according to claim 5.
The image processing apparatus characterized in that the setting means sets an amplification degree based on characteristics of background image data. The image processing apparatus according to claim 7.
The setting means sets the degree of amplification based on an average level of luminance values and a maximum level of luminance values among the characteristics of background image data. In an image processing method for outputting a blurred background image, comprising a blur processing step for blurring input background image data,
In the blurring process, when the value of the background image data in each pixel is high, a conversion step of amplifying the background image data in the pixel;
An image processing method comprising: an inverse conversion step for performing an inverse conversion of the conversion means. In an image processing method for outputting a blurred background image, comprising a blur processing step for blurring input background image data,
The blur processing step includes a conversion step of amplifying background image data in each pixel when the luminance value in each pixel is high;
An image processing method comprising: an inverse conversion step for performing an inverse conversion of the conversion means. The image processing method according to claim 9 or 10,
After the conversion step, filter processing is performed,
An image processing method comprising performing the inverse transformation step after performing the filtering process. The image processing method according to any one of claims 9 to 11,
The image processing method according to claim 1, wherein the conversion step further includes an image data amount expansion function for performing conversion for expanding an image data amount per pixel. The image processing method according to any one of claims 9 to 11,
An image processing method characterized in that the amplification degree of the conversion step is variable depending on the setting. The image processing method according to claim 13.
The image processing method according to claim 1, wherein the setting is based on a shooting mode. The image processing method according to claim 13.
The image processing method according to claim 1, wherein the setting is based on characteristics of background image data. The image processing apparatus according to claim 15, wherein
The setting is based on an average level of luminance values and a maximum level of luminance values among the characteristics of background image data.   The image processing apparatus according to claim 1, wherein the image processing apparatus is a digital camera.