JP5094219B2 - Image processing apparatus, image processing method, program, recording medium, and integrated circuit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize an image processor enabling a user to designate a degree of depth feeling with single operation. <P>SOLUTION: The image processor 100 includes an image data input part 200, a depth data input part 210, a depth degree input part 220, a depth feeling correction part 230, and an image data output part 240. The image processor 100 controls a gain of depth feeling correction in accordance with depth data and a depth degree, so that the user can freely adjust the degree of the depth feeling. The process of the degree of the depth feeling achieves both of near view process and distant view process, and adjustment is carried out with a one-dimensional parameter. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to an image processing technique for displaying a high-quality video on a video display device such as a TV, and to a technique for enhancing a sense of depth in a two-dimensional video.

As a conventional depth emphasis emphasis technique, there is a method of visually increasing a difference between an object and a surrounding background by clarifying a difference between a strong portion and a weak portion of a contour component (see, for example, Patent Document 1). ). FIG. 2 shows a configuration diagram of an image processing apparatus (three-dimensional expression circuit) 900 that realizes the conventional depth feeling correction technique described in Patent Document 1. In FIG.
In the image processing apparatus 900 shown in FIG. 2, data (input data) input from the data input terminal is sent to the averaging circuit 101 and the contour enhancement processing unit 140. The averaging circuit 101 averages input data, and the averaged data is output to the selector 103 via the delay circuit 102.
The contour enhancement processing unit 140 includes a delay circuit 104, a contour extraction unit 105, a contour correction processing unit 106, and an addition circuit 108.

First, the contour extraction unit 105 extracts a contour portion (contour component) using a digital filter, and outputs the result to the contour correction processing unit 106 and the threshold processing unit 107. The threshold processing unit 107 outputs a signal for determining selection of the selector 103 to the selector 103 based on the threshold setting signal from the threshold setting signal input terminal 120. The original signal (signal corresponding to the input data) delayed by the delay circuit 104 and the contour component gain-adjusted by the contour correction processing unit 106 are added by the adder circuit 108, and the added signal is sent to the selector 103. Is output.
The selector 103 selects one of the two input signals, that is, the averaged data output from the delay circuit 102 and the contour enhancement data output from the adder circuit 108, as a selection signal from the threshold processing unit 107. And output to the output terminal 130.
In the image processing apparatus 900 having such a configuration, the difference between the strong outline portion that is a close view and the weak portion that is a distant view on the image is adaptively selected between the edge enhancement signal and the averaged signal. You can clarify and make the image look three-dimensional.
JP 2004-159148 A (page 9, FIG. 1)

However, in the conventional image processing apparatus 900, when the threshold is changed by the threshold setting signal, the degree of alternative selection between the averaged data and the contour enhancement data is changed. For example, in the image processing apparatus 900, when the threshold value is lowered, the area determined to be an image to be contour-enhanced increases, and the size (ratio) of the emphasized area and the averaged area changes. However, the conventional technique does not add a change corresponding to the above change to each process for changing the degree of the feeling of depth generated by the difference in the degree of edge enhancement between the emphasized data and the averaged data.
Therefore, when depth emphasis is considered as a TV image quality adjustment function, there is a problem that the user cannot adjust the degree of depth according to his / her preference.
The present invention solves the above-described conventional problems, and provides an image processing apparatus, an image processing method, a program, a recording medium, and an integrated circuit capable of adjusting the degree of depth by an operation using a one-dimensional parameter. With the goal.

A first invention is an image processing apparatus including an image data input unit, a depth data input unit, a depth degree input unit, and a depth feeling correction unit. In the image data input unit, an image data signal capable of forming an image composed of pixels is input. In the depth data input unit, depth data corresponding to the image data signal is input. The depth degree input unit receives a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal. The depth feeling correction unit performs depth feeling correction on the image data signal based on the one-dimensional signal.
In this image processing device, a depth degree input unit inputs a one-dimensional signal for controlling a feeling of depth, and a depth correction unit executes a depth correction document process on the image data signal based on the one-dimensional signal. Is done.
As a result, the user can easily perform the depth sensation correction on the image data signal by inputting the one-dimensional signal from the depth degree input unit by operating the one-dimensional parameter. Further, the degree of depth in an image processed by this image processing apparatus can be easily adjusted to suit the user's preference by a single operation of the user.
Here, the “depth data corresponding to the image data signal” may be each pixel (or an area (block) made up of a plurality of pixels) constituting a two-dimensional image formed by the image data signal. .)), For example, for a pixel whose position (pixel position) on the two-dimensional image is (i, j), the depth associated with the pixel position (i, j). This is the case for data. The “depth data” is a value set for each pixel or each block obtained by dividing the screen into a plurality of blocks. For example, in the case of a two-dimensional image obtained by photographing an actual three-dimensional space with a camera, This indicates the degree of distance between the actual position in the three-dimensional space corresponding to the pixel (or block) on the two-dimensional image and the position in the three-dimensional space of the camera when the two-dimensional image is acquired. .

2nd invention is 1st invention, Comprising: The depth feeling correction | amendment part has a contour correction gain setting part, a contour correction part, a blur correction gain setting part, a blur correction part, and a selection part. . The contour correction gain setting unit sets the contour correction gain based on the depth data and the one-dimensional signal. The contour correcting unit performs contour correction on the image data signal based on the contour correction gain. The blur correction gain setting unit sets a blur correction parameter based on the depth data and the one-dimensional signal. The blur correction unit performs blur correction on the image data signal based on the blur correction parameter. The selection unit selects one of the image data signal corrected by the contour correction unit and the image data signal corrected by the blur correction unit based on the depth data and the one-dimensional signal.
In this image processing apparatus, it is possible to perform contour correction processing with the contour correction gain set by the contour correction gain setting unit based on the depth data and the one-dimensional signal, and based on the depth data and the one-dimensional signal, The blur correction process can be performed using the blur correction parameters set by the blur correction gain setting unit. Then, the selection process in the selection unit is controlled based on the depth data and the one-dimensional signal.
Thereby, in this image processing apparatus, the intensity (degree) of the contour correction process and the blur correction process performed on the image data signal can be easily adjusted based on the one-dimensional signal.

3rd invention is 2nd invention, Comprising: The outline correction gain setting part, as the value of the one-dimensional signal increases, the outline correction gain for the foreground area on the image where the value of the depth data is small, and the depth data The contour correction gain is set so that the difference from the contour correction gain for a distant view area on an image with a large value of is large.
Thereby, when the user increases the value of the one-dimensional signal, the outline of the foreground region on the image (processed image) processed by the image processing apparatus is emphasized, and the depth effect is enhanced in the processed image.
Note that the “foreground region” refers to, for example, a depth data distribution for all pixels forming an image. In the depth data distribution, the depth data is a predetermined threshold T1 (the smaller the depth data value, the closer the foreground region is). This is an area on the image including pixels having a smaller value. The “distant view area” is, for example, a distribution of depth data for all the pixels forming the image. In the depth data distribution, this is an area on the image including pixels whose depth data has a value larger than a predetermined threshold T2 (> T1). Note that the threshold values T1 and T2 of the depth data may be the same value. Further, the “near view area” and the “far view area” may be determined relatively.

The fourth invention is the second invention, wherein the blur correction gain setting unit does not execute blur processing between depth data Z0 and Z2 (> Z0) regardless of the value of the one-dimensional signal. As described above, when the blur correction parameter is set and the depth data exceeds Z2, as the value of the one-dimensional signal increases, the degree of blur processing when the depth data value is Z2 and the value of the depth data The blur correction parameter is set so that the difference from the degree of blur processing when Z1 (> Z2) is large.
As a result, the greater the value of the one-dimensional signal, the greater the degree of blur correction processing for the pixels in the distant area where the depth data value is large, so that the distant area on the image is visually deeper. It can be made to feel, and the sense of depth can be enhanced in the processed image.

5th invention is 2nd invention, Comprising: When the value of a one-dimensional signal is below 1st value ZT, a selection part selects the image data signal by which the outline correction | amendment correction | amendment was carried out, and is 1D When the value of the signal is greater than the first value ZT and the value of the depth data is equal to or less than the first depth data value Z2, the image data signal whose contour has been corrected by the contour correction unit is selected, and one-dimensional When the value of the signal is greater than the first value ZT and the value of the depth data is greater than the first depth data value Z2, the image data signal that has been subjected to blur correction by the blur correction unit is selected.
Thereby, it is possible to effectively enhance the sense of depth in the processed image simply by changing the value of the one-dimensional signal.

6th invention is 2nd invention, Comprising: The depth correction | amendment part is based on the saturation correction gain setting part which sets saturation correction gain based on depth data and a one-dimensional signal, and saturation correction gain. And a saturation correction unit for performing saturation correction on the image data signal.
As a result, the intensity (degree) of the saturation correction processing applied to the image data signal can be easily adjusted simply by changing the value of the one-dimensional signal.
The seventh invention is the sixth invention, wherein the saturation correction gain setting unit has a saturation correction gain for a foreground region on an image having a smaller depth data value as the value of the one-dimensional signal increases, The saturation correction gain is set so that the difference from the saturation correction gain for a distant view area on an image having a large depth data value becomes large.
Accordingly, as the value of the one-dimensional signal increases, the degree of saturation correction processing for the pixels in the foreground area where the depth data value is small increases (because the saturation is emphasized). The pixels in the foreground area can be made to feel more vivid, and the sense of depth can be enhanced in the processed image.

8th invention is 1st invention, Comprising: A depth feeling correction | amendment part sets the contrast correction gain setting part which sets a contrast correction gain based on depth data and a one-dimensional signal, An image based on a contrast correction gain A contrast correction unit that performs contrast correction on the data signal is provided.
Thereby, the intensity (degree) of the contrast correction processing applied to the image data signal can be easily adjusted only by changing the value of the one-dimensional signal, and the depth feeling can be effectively enhanced.

A ninth invention is the eighth invention, wherein the contrast correction gain setting unit increases the contrast correction gain for the foreground region on the image having a smaller depth data value and the depth data as the value of the one-dimensional signal increases. The contrast correction gain is set so that the difference from the contrast correction gain for a distant view area on an image having a large value of is large.
As a result, the larger the value of the one-dimensional signal, the stronger the degree of contrast correction processing for the pixels in the foreground area where the depth data value is smaller (because the contrast is enhanced). These pixels can be made to feel more clearly, and a sense of depth can be enhanced in the processed image.

A tenth invention is the first invention, wherein the depth sensation correction unit is configured to set a shadow correction gain based on the depth data and the one-dimensional signal, and an image based on the shadow correction gain. A shadow correction unit that performs shadow correction on the data signal is provided.
As a result, the intensity (degree) of the shadow correction processing applied to the image data signal can be easily adjusted simply by changing the value of the one-dimensional signal, and the sense of depth can be effectively enhanced.
11th invention is 9th invention, Comprising: A shadow correction gain setting part is a shadow correction gain with respect to the foreground area on an image with a small value of depth data, and depth data, so that the value of a one-dimensional signal becomes large. The shadow correction gain is set so that the difference from the shadow correction gain for the distant view area on the image having a large value of becomes large.
As a result, the greater the value of the one-dimensional signal, the greater the degree of shadow correction processing for pixels in the foreground area where the depth data value is smaller (because the shadow is enhanced). The shading feeling of the pixels is enhanced, and the sense of depth can be enhanced in the processed image.

The twelfth invention is the first invention, wherein when the one-dimensional signal is equal to or less than a predetermined value, the depth sensation correction unit performs only the foreground correction processing on the image data signal, and the one-dimensional signal Is larger than the predetermined value, the near-field correction process and the far-field correction process are performed on the image data signal.
As a result, until the one-dimensional signal reaches a predetermined value, the foreground correction processing is performed on the pixels in the foreground area where the depth data value is small, and the one-dimensional signal has the predetermined value. If it exceeds the threshold value, it is possible to further enhance the sense of depth in the processed image by performing a process for emphasizing that the pixel in the distant view area having a large depth data value is a distant view.
Here, the “correction process for near view” is, for example, a process for enhancing contrast, a process for enhancing saturation, or a process for enhancing shadows. Further, the “distant view process” includes, for example, a process for suppressing contrast, a process for suppressing saturation, and a process for suppressing shadows.

The thirteenth invention is the first invention, in which the depth sensation correcting unit increases the correction amount for the image data signal corresponding to the pixel of the nearest scene on the image and the image on the image as the value of the one-dimensional signal increases. The difference with the correction amount for the image data signal corresponding to the farthest view pixel is enlarged.
Thereby, a sense of depth can be effectively enhanced in the image processed by the image processing apparatus.
The fourteenth invention is the first invention, wherein the depth sensation correction unit includes a correction amount for the image data signal corresponding to the pixel of the nearest scene on the image, and the image on the image, regardless of the value of the one-dimensional signal. The difference from the correction amount for the image data signal corresponding to the farthest view pixel is made constant.
Thereby, a sense of depth can be effectively enhanced in the image processed by the image processing apparatus.

A fifteenth aspect of the invention is the twelfth aspect of the invention, wherein the depth sensation correction unit performs a contour enhancement process as a foreground correction process for the image data signal, and a blur correction as a distant view correction process for the image data signal. Process.
A sixteenth aspect of the invention is the twelfth aspect of the invention, in which the depth sensation correction unit performs a saturation enhancement process as a foreground correction process for the image data signal, and a distant view correction process for the image data signal. Saturation processing is performed.
In a seventeenth aspect based on the first aspect, the depth sensation correction unit performs a correction process on a predetermined feature amount of the image based on the one-dimensional signal.
Here, the “feature amount” is a physical quantity that can be considered for each pixel or region in an image, for example, brightness (luminance), brightness, saturation, hue, contrast, brightness contrast, for example. Color contrast and the like.

The eighteenth invention is the seventeenth invention, wherein the depth sensation correction unit corrects an image data signal forming an image including a first area and a second area that can be distinguished by depth data. The image data signal in the first area is corrected by controlling the degree of enhancement of the correction process, and the image data signal in the second area is corrected. Correction processing is performed by controlling the degree of suppression of processing.
In a nineteenth aspect based on the eighteenth aspect, the depth sensation correction unit controls the enhancement degree of correction processing for the image data signal in the first region based on the one-dimensional signal. Correction processing is performed, and the correction processing is performed on the image data signal in the second region by controlling the degree of suppression of the correction processing.
Accordingly, the correction enhancement processing is performed on the image data signal in the first region and the correction suppression processing is performed on the image data signal in the second region in conjunction with the one-dimensional signal. Can do.

The twentieth invention is the eighteenth invention, wherein the depth sensation correction unit performs edge enhancement processing on the image data signal in the first region, and applies to the image data signal in the second region. Performs a blur correction process on the outline.
The twenty-first invention is the eighteenth invention, wherein the depth sensation correction unit performs a saturation enhancement process on the image data signal in the first area, and performs an image data signal in the second area. In this case, desaturation processing is performed.
The twenty-second invention is the eighteenth invention, wherein the depth sensation correction unit performs a contrast enhancement process on the image data signal in the first area, and performs an image data signal in the second area. Performs contrast suppression processing.
The twenty-third invention is the eighteenth invention, wherein the depth sensation correction unit performs a shadow enhancement process on the image data signal in the first region, and performs an image data signal in the second region. Performs shading suppression processing.

A twenty-fourth aspect is the seventeenth aspect, wherein the depth correction unit corrects an image data signal forming an image including a first area and a second area that can be distinguished by depth data. A first correction process is performed on the image data signal in the first area, and a second different from the first correction process is performed on the image data signal in the second area. Perform correction processing.
The twenty-fifth invention is the twenty-fourth invention, wherein the depth sensation correction unit is one of four correction processes including a contour correction process, a saturation correction process, a contrast correction process, and a shadow correction process as the first correction process. One is selected, the selected first correction process is performed, and the second correction process is selected from three correction processes excluding those selected as the first correction process from the four correction processes, and the selected second correction is performed. Process.

The twenty-sixth invention is the twenty-fourth invention, wherein the depth sensation correction unit controls the intensity of the first correction process and the intensity of the second correction process based on the one-dimensional signal.
Thereby, the intensity | strength of a 1st correction process and the intensity | strength of a 2nd correction process can be controlled in response to a one-dimensional signal.
A twenty-seventh invention is an image processing system including an image processing device and a controller. The image processing apparatus includes an image data input unit, a depth data input unit, a depth degree input unit, and a depth feeling correction unit. In the image data input unit, an image data signal capable of forming an image composed of pixels is input. In the depth data input unit, depth data corresponding to the image data signal is input. The depth degree input unit receives a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal. When the value of the one-dimensional signal is less than or equal to a predetermined value, the depth correction unit performs contour enhancement processing on the image data signal in the foreground area on the image, and the value of the one-dimensional signal is greater than the predetermined value Then, edge enhancement processing is performed on the image data signal in the foreground region on the image, and blur correction processing is performed on the image data signal in the far view region on the image. The controller includes an operation unit that allows a user to input a one-dimensional signal to a depth degree input unit of the image processing apparatus by performing a one-dimensional operation.
Accordingly, in this image processing system, the contour enhancement processing and blur correction processing in the image processing device can be selected by a single operation by the user to the operation unit of the controller, and the contour enhancement in the image processing device can be selected. The processing intensity of the processing and the blur correction processing can be adjusted.

A twenty-eighth aspect of the invention is an image processing method comprising an image data input step, a depth data input step, a depth degree input step, and a depth feeling correction step. In the image data input step, an image data signal capable of forming an image composed of pixels is input. In the depth data input step, depth data corresponding to the image data signal is input. In the depth degree input step, a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal is input. In the depth feeling correction step, depth feeling correction is performed on the image data signal based on the one-dimensional signal.
As a result, an image processing method having the same effect as that of the first invention can be realized.

A twenty-ninth invention is a program for causing a computer to function as an image data input unit, a depth data input unit, a depth degree input unit, and a depth feeling correction unit. In the image data input unit, an image data signal capable of forming an image composed of pixels is input. In the depth data input unit, depth data corresponding to the image data signal is input. The depth degree input unit receives a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal. The depth sensation correction unit performs depth sensation correction on the image data signal based on the one-dimensional signal.
As a result, it is possible to realize a program that exhibits the same effects as those of the first invention.
A thirtieth aspect of the present invention is a computer-readable recording medium that records a program for causing a computer to function as an image data input unit, a depth data input unit, a depth degree input unit, and a depth feeling correction unit. In the image data input unit, an image data signal capable of forming an image composed of pixels is input. In the depth data input unit, depth data corresponding to the image data signal is input. The depth degree input unit receives a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal. The depth sensation correction unit performs depth sensation correction on the image data signal based on the one-dimensional signal.
Thereby, it is possible to realize a computer-readable recording medium in which a program having the same effect as that of the first invention is recorded.

A thirty-first invention is an integrated circuit including an image data input unit, a depth data input unit, a depth degree input unit, and a depth feeling correction unit. In the image data input unit, an image data signal capable of forming an image composed of pixels is input. In the depth data input unit, depth data corresponding to the image data signal is input. The depth degree input unit receives a one-dimensional signal for controlling the sense of depth in an image formed by the image data signal. The depth sensation correction unit performs depth sensation correction on the image data signal based on the one-dimensional signal.
Thus, an integrated circuit that exhibits the same effect as that of the first invention can be realized.

According to the image processing device, the image processing method, the program, the recording medium, and the integrated circuit of the present invention, when adjusting the feeling of depth as image quality adjustment, the degree of the feeling of depth can be adjusted by operating one-dimensional parameters. it can.
Furthermore, according to the image processing apparatus, the image processing method, the program, the recording medium, and the integrated circuit of the present invention, the user can set the same parameter by the input unit, thereby reducing the degree of depth feeling by a single operation of the user. It can be easily adjusted to suit user preferences.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
<1: Configuration of image processing apparatus>
FIG. 1 shows a block diagram of an image processing apparatus 100 according to the first embodiment of the present invention.
The image processing apparatus 100 according to the present embodiment mainly includes an image data input unit 200 to which an image (image signal (image data)) subjected to depth enhancement processing is input, and an image signal input to the image processing apparatus 100. Depth data input unit 210 to which depth data corresponding to (input image signal) is input; Depth level input unit 220 for inputting a depth level desired by the user; Depth data and depth input from depth data input unit 210 Based on the degree of depth input from the degree input unit 220, a depth sensation correction unit 230 that performs a depth sensation correction on the input image signal, and an image signal corrected by the depth sensation correction unit 230 is output in a predetermined format. Image data output unit 240.

The image data input unit 200 receives an image (image signal (image data)) on which depth enhancement processing is performed, and outputs the input image signal (input image signal) to the depth sensation correction unit 230.
The depth data input unit 210 receives the depth data corresponding to the input image signal and outputs the input depth data to the depth feeling correction unit 230. Here, the “depth data corresponding to the input image signal” is each pixel (or may be an area (block) made up of a plurality of pixels) constituting a two-dimensional image formed by the input image signal. For example, for a pixel whose position (pixel position) on the two-dimensional image of the pixel is (i, j), the depth data associated with the pixel position (i, j) is This is the case. The “depth data” is a value set for each pixel or each block obtained by dividing the screen into a plurality of blocks. For example, in the case of a two-dimensional image obtained by photographing an actual three-dimensional space with a camera, This indicates the degree of distance between the actual position in the three-dimensional space corresponding to the pixel (or block) on the two-dimensional image and the position in the three-dimensional space of the camera when the two-dimensional image is acquired. .

For example, the depth data is set to take an integer value of “0” to “7”, and the position in the three-dimensional space of the pixel A (pixel position (i1, j1)) on the two-dimensional image and the three-dimensional camera When the distance to the position in the space is smaller than the predetermined value L1 (close view), the depth data of the pixel A (pixel position (i1, j1)) is set to “0” (value indicating close view). . When the distance between the position in the three-dimensional space of the pixel B (pixel position (i2, j2)) on the two-dimensional image and the position in the three-dimensional space of the camera is larger than the predetermined value L2 (distant view) , The depth data of the pixel B (pixel position (i2, j2)) is “7” (a value indicating a distant view). In this way, the depth data for each pixel (or each region) may be determined.
Further, the value of the depth data may be data acquired in advance by, for example, a range finder (a distance meter linked to the photographing lens). The value of the depth data may be acquired by a method such as stereo measurement using a plurality of images (for example, a method using stereo matching).

The depth degree input unit 220 receives the user's favorite depth degree (depth degree data) and outputs the input depth degree to the depth sensation correction unit 230.
The depth sensation correction unit 230 performs depth sensation correction on the input image signal based on the depth data input from the depth data input unit 210 and the depth level input from the depth level input unit 220.
As shown in FIG. 3, the depth sensation correction unit 230 includes a contour correction unit 231 that performs contour correction on the input image signal, a blur correction unit 232 that performs blur correction on the input image signal, and a depth data input unit. Based on the depth data output from 210 and the depth level output from the depth level input unit 220, one of the image signal output from the contour correction unit 231 and the image signal output from the blur correction unit 232 is selected. And a saturation correction unit 233 that performs saturation correction on the image signal output from the selection unit 237.

Further, the depth sensation correction unit 230, based on the depth data and the depth degree, respectively, a contour correction gain setting unit 234 that sets a contour correction gain, a blur correction gain setting unit 235 that sets a blur correction gain, and saturation. A saturation correction gain setting unit 236 for setting the correction gain.
The contour correction unit 231 receives the input image signal output from the image data input unit 200 and the contour correction gain output from the contour correction gain setting unit 234, and based on the contour correction gain, Contour correction processing is performed. The contour correction unit 231 outputs the image signal subjected to the contour correction processing to the selection unit 237.
The blur correction unit 232 receives the input image signal output from the image data input unit 200 and the blur correction gain output from the blur correction gain setting unit 235, and blurs the input image signal based on the blur correction gain. Make corrections. The blur correction unit 232 outputs the image signal subjected to the blur correction to the selection unit 237.

The selection unit 237 outputs the depth data output from the depth data input unit 210 and the depth level output from the depth level input unit 220, and the image signal output from the contour correction unit 231 and the blur correction unit 232. Based on the depth data and the depth level, one of the output from the contour correction unit 231 and the output from the blur correction unit 232 is selected and output to the saturation correction unit 233.
The saturation correction unit 233 receives the image signal output from the selection unit 237 and the saturation correction gain output from the saturation correction gain setting unit 236, and is output from the selection unit 237 based on the saturation correction gain. The image signal is subjected to saturation correction and output to the image data output unit 240.
<2: Operation of Image Processing Device>
The operation of the image processing apparatus 100 configured as described above will be described.

The depth correction unit 230 includes three types of image data (image signal) from the image data input unit 200, depth data from the depth data input unit 210, and a depth level specified by the user from the depth level input unit 220. Signal is input.
Here, the “depth data” is a value set for each pixel or each block obtained by dividing the screen into a plurality of values, and has values such as “0” to “7”, for example. These values are data acquired in advance by a method such as a range finder, for example. In addition, you may acquire by methods, such as stereo measurement by several images.
Here, the “degree of depth” is a one-dimensional value (one-dimensional data) determined by a user operation, display resolution, resolution of image data (image signal), and the like. In the depth degree input unit 220, the one-dimensional value (depth degree) is input by the user. Then, in the image processing apparatus 100, for example, adjustment of the correction amount of the depth feeling desired by the user is executed by using the one-dimensional value (about the depth). Typically, the one-dimensional value (about the depth) is adjusted by a TV remote control or an adjustment knob mounted on the TV itself. In the case of the remote control, first, the user selects an image quality adjustment screen, and, for example, a GUI displayed with a one-dimensional bar as shown in FIG. 4 is presented. The user operates the buttons on the remote control and slides the inverted triangle arrow shown in Fig. 4 to the left to adjust the depth feeling. By sliding the inverted triangle arrow to the right, the depth feeling is emphasized. Adjustment to be performed is executed. In this way, the user operates the remote controller or the like to adjust the depth feeling level to a desired position by a single operation. In this way, the depth degree is input from the depth degree input unit 220.

The image data output unit 240 receives the output of the depth sensation correction unit 230 and outputs image data (image signal) with the depth sensation corrected.
FIG. 3 is a block diagram showing the configuration of the depth sensation correction unit 230.
The depth correction unit 230 includes image data (input image signal) input by the image data input unit 200, depth data input by the depth data input unit 210, and a depth level input by the depth level input unit 220. These three types of signals are input. The image data (image signal) is input to the contour correction unit 231 and the blur correction unit 232.
The depth data and the depth specified by the user are input to the contour correction gain setting unit 234, the blur correction gain setting unit 235, the saturation correction gain setting unit 236, and the selection unit 237, respectively.

FIG. 5 shows details of an example of the contour correction unit 231.
As shown in FIG. 5, the contour correction unit 231 is mainly configured by a delay unit 2311, a high-pass unit 2312, a gain adjustment unit 2313, and an addition unit 2314.
The image signal input from the image data input unit 200 is delayed by the delay unit 2311 in order to match the time with the contour correction signal (for timing adjustment), and is output to the adder unit 2314. The image signal input from the image data input unit 200 is subjected to processing (for example, HPF (High Pass Filter) processing) by the high pass unit 2312. The image signal processed by the high-pass unit 2312 is input to the gain adjustment unit 2313, and gain adjustment is performed by the gain adjustment unit 2313 using the gain set by the contour correction gain setting unit 234 and output to the addition unit 2314. The The adder 2314 adds the image signal (image signal corresponding to the original image) output from the delay unit 2311 and the image signal (outline enhancement signal) output from the gain adjuster 2313, and outputs the result to the selector 237. The In other words, the image signal acquired by the addition by the addition unit 2314 is output to the selection unit 237 as a signal whose contour is corrected and sharpness is enhanced.

  Returning to FIG. 3, the blur correction unit 232 uses the parameter (blur correction gain) σ set by the blur correction gain setting unit 235, and the image data (image signal) is expressed by a Gaussian function represented by the following equation: Blurred (subjected to blur correction processing).

Here, (x0, y0) is a pixel position to be processed (a pixel position on a two-dimensional image formed by an image signal), and is averaged over a square range of 2N + 1 pixels in the horizontal direction and 2M + 1 pixels in the vertical direction. It becomes.
The image signal subjected to the blur correction process in this way is output to the selection unit 237.
The selection unit 237 selects either the output of the contour correction unit 231 or the output of the blur correction unit 232 according to the depth data output from the depth data input unit 210 and the depth level output from the depth level input unit 220. And output to the saturation correction unit 233.
In the selection unit 237, the output of the contour correction unit 231 is selected and output to the saturation correction unit 233 until the depth level reaches a predetermined value ZT. When the depth degree exceeds a predetermined value ZT and the depth data value from the depth data input unit 210 is less than Z2, the output of the contour correction unit 231 is selected, and the depth data value is greater than Z2. When it is a distant view from Z2, the output of the blur correction unit 232 is selected and output to the saturation correction unit 233.

As shown in FIG. 6, the saturation correction unit 233 mainly includes an HSV conversion unit 2331, a delay unit 2332, a saturation addition unit 2333, and an inverse HSV conversion unit 2334.
The image signal output from the selection unit 237 is separated into an H (hue) signal, an S (saturation) signal, and a V (luminance) signal by an HSV conversion unit 2331, and the H (hue) signal and the V (luminance) signal are The signal is output to the inverse HSV conversion unit 2334 via the delay unit 2332.
The S (saturation) signal separated by the HSV conversion unit 2331 is output to the saturation addition unit 2333, the saturation is added, and is output to the inverse HSV conversion unit 2334.
In the inverse HSV conversion unit 2334, the H (hue) signal and V (brightness) signal output from the delay unit 2332 and the S (saturation) signal added with the saturation by the saturation addition unit 2333 are converted into the original HSV conversion unit 2334. An image signal in a format (image signal before being subjected to HSV conversion by the HSV conversion unit 2331) is reconstructed (inverse HSV conversion) and output to the image data output unit 240. Here, the H (hue) signal output from the delay unit 2332 is a signal having the same H (hue) as the current signal (image signal output from the selection unit 237), and the V signal output from the delay unit 2332. The (luminance) signal is a signal having the same V (luminance) as the current signal (image signal output from the selection unit 237).

In this way, in the image processing apparatus 100, the depth sensation correction is performed by contour correction, blur correction, and saturation correction.
(Gain setting method)
Next, each gain setting method will be described.
First, the contour correction gain setting unit 234 sets a gain to be input to the gain adjustment unit 2313 constituting the contour correction unit 231 from the depth data and the depth degree.
The gain settings in the contour correction gain setting unit 234, the blur correction gain setting unit 235, and the saturation correction gain setting unit 236 are, for example, according to a table (LUT (Look Up Table)) that realizes characteristics as shown in FIG. Done.
<Outline correction gain>
First, setting of the contour correction gain by the contour correction gain setting unit 234 will be described.

Depth data-contour correction gain characteristics used for setting the contour correction gain will be described.
In the graph shown in FIG. 7A, the horizontal axis represents the depth data value, and the vertical axis represents the contour correction gain.
When the input from the depth degree input unit 220 is “0”, the gain is “0” in all the depth data as in the characteristic shown in the graph A of FIG.
If the input from the depth degree input unit 220 is increased from “0”, the depth data-contour correction gain characteristic is inclined between Z0 and Z1, for example, as shown in the graph B of FIG. (The slope is not “0”. However, only the slope “0” of the straight line shown in the graph A is included.) The graph (characteristic) is obtained.

In the depth data-contour correction gain characteristic, if the depth degree further increases, the slope of the straight line (which may be a curve) indicating the depth data-contour correction gain characteristic gradually decreases, and the depth degree is predetermined. When the value ZT is reached, the characteristics shown in the graph C are obtained, and even if the depth is increased, no further decrease is made unless the predetermined value ZT is exceeded.
In the depth data-contour correction gain characteristic, when the depth degree exceeds a predetermined value ZT, the depth data whose depth correction value is “0” is kept from Z1 to Z2 while the gain at the depth data Z0 remains unchanged. (Changes to the characteristics of graph D).
FIG. 7B is a view of the characteristics shown in FIG. 7A from another viewpoint. A straight line (curve) showing the depth degree on the horizontal axis and the depth data-contour correction gain characteristics on the vertical axis. It is the graph which took the inclination (change amount (differential amount) with respect to depth data of an outline correction gain). Note that the vertical axis in FIG. 7B indicates the slope of the straight line (curve) indicating the depth data-contour correction gain characteristic in a portion where the slope of the straight line (curve) indicating the depth data-contour correction gain characteristic is not “0”. (Including only the slope “0” of the straight line shown by graph A in FIG. 7A).

As shown in FIG. 7B, the inclination of the contour correction gain decreases monotonously as the depth level increases.
As described above, the contour correction gain is set such that the difference between the contour enhancement degree in the farthest region and the contour enhancement degree in the nearest region increases with a change in the depth which is one-dimensional data. .
<Blur correction gain>
Next, the blur correction gain setting unit 235 acquires a blur correction gain parameter with reference to a table that realizes characteristics as illustrated in FIG. 8A, and the acquired blur correction gain parameter is the blur correction unit. Output to H.232. The blur correction unit 232 executes the blur correction process with the blur correction gain determined by the blur correction parameter. The process in the blur correction unit 232 is a process corresponding to (Equation 1).

In the graph shown in FIG. 8, the horizontal axis represents depth data, and the vertical axis represents the value of the correction gain parameter σ used in (Equation 1).
The depth data-blur correction gain parameter characteristic is the characteristic of the graph E until the depth degree reaches a predetermined value ZT, and the value of the blur correction gain parameter σ is "0" for all depth data. Become. Therefore, in this case, the value “0” of the blur correction gain parameter σ is output to the blur correction unit 232, and the blur correction unit 232 receives an input of σ = 0 as shown in (Formula 1). The input image data (image signal) is output as it is without doing anything.
As the degree of depth input from the degree-of-depth input unit 220 reaches the predetermined value ZT and the degree of depth becomes larger than the predetermined value ZT, the parameter characteristic for depth data-blur correction gain is represented by the graph E in FIG. The characteristics shown in FIG. 6 change to the characteristics shown in the graph F and the characteristics shown in the graph G. For example, when the depth data-blur correction gain parameter characteristic is determined by the characteristic shown in the graph F of FIG. 8, the value of the finite blur correction gain parameter σ determined by the graph F is input to the blur correction unit 232. Is output.

Then, in proportion to the depth degree, the slope of the straight line indicating the depth data-blur correction gain parameter characteristic increases, and finally, the slope of the straight line having the characteristic shown in the graph G becomes the upper limit. Here, the value Z2 of the depth data is the same value as Z2 in the graph of FIG. 7 of the contour correction gain setting unit 234.
FIG. 8B is a view of the characteristics shown in FIG. 8A from another viewpoint. The horizontal axis indicates the depth, and the vertical axis indicates the depth data-blur correction gain parameter characteristics. It is the graph which took inclination (change amount (derivative amount) with respect to depth data of parameter σ for outline correction gain).
As shown in FIG. 8B, as the degree of depth increases, the slope of the straight line indicating the parameter characteristic for depth data-blur correction gain increases monotonously.

As described above, the contour correction gain parameter σ is determined by the blur enhancement degree in the farthest region and the nearest region on the two-dimensional image formed by the image signal according to the change of the depth which is one-dimensional data. The difference from the degree of blur enhancement is set to be large. That is, the blur correction gain setting unit 235 sets the contour correction gain parameter σ based on the depth data-blur correction gain parameter characteristic described above.
<Saturation correction gain>
Next, setting of the saturation correction gain by the saturation correction gain setting unit 236 will be described.
Depth data-saturation correction gain characteristics used for setting the saturation correction gain will be described.
The saturation correction gain setting unit 236 acquires a saturation gain with reference to a table that realizes the characteristics as illustrated in FIG. 9A, and outputs the acquired saturation gain to the saturation correction unit 233.

In the graph shown in FIG. 9A, the horizontal axis is the depth, and the vertical axis is the increase in saturation.
In FIG. 9A, when the depth degree is “0”, ΔS is “0” regardless of the depth data value Z (characteristic shown by the graph H). As the depth is increased, the slope of the depth data-saturation correction gain characteristic line (curve) between the depth data Z0 and Z1 increases as shown by the graph I in FIG. 9A. When the depth degree reaches a predetermined value ZT, the depth data-saturation correction gain characteristic becomes a characteristic indicated by the graph J. When the depth degree exceeds a predetermined value ZT, a saturation correction gain having a negative addition gain such as the characteristic indicated by the graph K in FIG. 9A is acquired by the saturation correction gain setting unit 236, and the saturation correction gain setting unit 236 acquires the saturation correction gain. Is output to the degree correction unit 233. When the depth is further increased, a saturation correction gain having a larger negative addition gain is acquired by the saturation correction gain setting unit 236 (for example, acquired by the LUT) as shown in the graph L, and the saturation correction is performed. Is output to the unit 233.

FIG. 9B is a diagram in which the characteristics shown in FIG. 9A are viewed from another point of view. The horizontal axis indicates the depth, and the vertical axis indicates the depth data ranging from Z0 to Z1. It is the graph which took the inclination of the straight line (it may be a curve) which shows depth data-saturation correction gain characteristic.
FIG. 9C is a diagram in which the characteristics shown in FIG. 9A are viewed from another viewpoint. The horizontal axis indicates the depth, and the vertical axis indicates the saturation correction gain when the depth data is Z1. It is the graph which took the value.
As shown in FIG. 9B, the slope of the straight line (curve) indicating the depth data-saturation correction gain characteristic monotonously decreases as the depth degree increases. In FIG. 9C, the value of the saturation correction gain when the depth data is Z1 is zero in the region where the depth is “weak” to “medium”, and the saturation correction gain is all positive (depth). When the data is less than Z1, the saturation correction gain takes a positive value, and when the depth data has a value greater than or equal to Z1, the saturation correction gain takes “0”. In the region where the depth is “medium” to “strong”, the saturation correction gain decreases monotonously in the negative region as the depth increases. That is, as the depth degree increases, the saturation correction gain takes a negative value and increases its absolute value.

As described above, in the image processing apparatus 100, the saturation enhancement processing and the saturation suppression processing are simultaneously performed in one frame image in accordance with the change of the depth level which is one-dimensional data.
By such a gain setting method, the image processing apparatus 100 increases the degree of the effect of contour correction and saturation correction as the depth is closer (the depth data value is smaller) until the depth degree reaches a predetermined value ZT. be able to.
Then, in the image processing apparatus 100, when the depth degree exceeds the predetermined value ZT, the blurring process and the saturation reduction process are performed on the distant area (distant view area) without changing the close area (distant view area) on the image. Added. In this way, the image processing apparatus 100 can further increase the degree of depth on the processed image by increasing the difference between the near-field region and the far-field region with respect to the degree of contour enhancement and saturation. The depth can be adjusted according to the user's subjectivity.

In this way, in the image processing apparatus 100, by increasing the depth level to a predetermined value, only the sense of depth can be increased without degrading the image quality, and the depth data is set to exceed the predetermined value. Thus, although the image quality of the processed image is deteriorated, the sense of depth can be further increased. That is, in the image processing apparatus 100, the degree of depth of the processed image can be increased or decreased by a simple operation that matches the user's subjectivity.
In the image processing apparatus 100 according to the present embodiment, depth sense correction is performed by contour correction, blur correction, and saturation correction, but it is not necessary to use all corrections. For example, in the image processing apparatus 100, the input image signal is subjected to a contour correction process and a saturation correction process that performs only a saturation enhancement process among the saturation corrections, and blur correction is performed on the input image signal. The process and the saturation suppression process of the saturation correction may not be executed.

FIG. 11 shows an example thereof. In this case, in the image processing apparatus 100, the depth data-contour emphasis correction gain characteristic straight line (curve) at the depth data Z = Z0 shown in FIG. The slope (the slope of the characteristic line when the depth data Z is from Z0 to Z2) is monotonously decreased, and when the depth is from “medium” to “strong”, the slope of the depth data—the contour characteristic correction gain characteristic line (curve) Is set to a constant value, only the edge enhancement is performed on the input image signal according to the depth data Z.
Further, in the image processing apparatus 100, the slope of the depth data-saturation correction gain characteristic line (curve) when the depth data Z = Z0 shown in FIG. 9A (the characteristic line where the depth data Z is from Z0 to Z1). As the depth degree increases (strong) from “weak” to “medium”, it decreases monotonically from “0”, and when the depth degree is larger than “medium”, depth data-saturation correction The slope of the gain characteristic straight line (curve) is set to a predetermined constant value.

  In the image processing apparatus 100, the saturation correction gain in the depth data Z = Z1 shown in FIG. 9A is set to “0” when the depth degree is “weak” to “medium”. As the depth degree increases from “medium”, control is performed so as to monotonously decrease from “0” toward a negative value. Accordingly, when the depth degree is strong, the saturation correction gain becomes a negative value in the distant view area in the depth data Z = Z1, and the distant feeling can be obtained by performing the saturation suppression process. Note that the saturation correction gain at depth data Z = Z1 in FIG. 9A is a predetermined positive definite value, and decreases monotonously from the predetermined positive definite value toward “0” as the depth degree increases from “medium”. You may control to do. In this case, when the depth degree is strong, the saturation correction process is not performed in the distant view area in the depth data Z = Z1. That is, in this case, only the saturation correction processing is performed on the region where the depth data is smaller than Z1 (the saturation suppression processing is performed even if the depth data value is larger than Z1). Never).

As described above, in the image processing apparatus 100, two different emphasis processes are performed to give the image a depth effect (an effect of increasing the depth sensation), and two depth emphasis values input by the user alone are used. Correction processing can be controlled simultaneously.
However, in the image processing apparatus 100, when the depth sensation correction described here is performed by contour correction, blur correction, and saturation correction, as shown in FIG. 9 and FIG. Contour correction and saturation enhancement processing are used together until the degree reaches a predetermined value (here, the value corresponding to the depth degree “medium”). If the depth exceeds the predetermined value, blur correction and saturation suppression are performed. Further processing is more preferable from the viewpoint of the depth effect. This is because the depth sensation that the user feels can be further increased.

In the image processing apparatus 100 according to the present embodiment, the contour correction and the saturation correction are adopted as the image quality correction. However, the present invention is not limited to this, and different gains are used for the foreground area and the distant area on the image. Any processing that can increase the sense of depth by applying may be employed.
(Modification 1)
Next, Modification 1 of the image processing apparatus 100 according to the present embodiment will be described.
Modification 1 is characterized in that contrast correction processing is performed on an image signal. Since other than that is the same as that of the above-mentioned embodiment, description is abbreviate | omitted.
Note that the image processing apparatus according to the first modification is realized by replacing the depth feeling correction unit 230 with the depth feeling correction unit 2301 illustrated in FIG. 13 in the image processing apparatus 100 of the first embodiment.

The image processing apparatus according to the first modification may be realized by replacing the depth feeling correction unit 230 with the depth feeling correction unit 2302 illustrated in FIG. 14 in the image processing apparatus 100 according to the first embodiment.
Furthermore, in the image processing apparatus according to the first modification, in the image processing apparatus 100 of the first embodiment, the contour correction unit 231 of the depth sensation correction unit 230 is replaced with a contrast correction unit 301, and the contour correction gain setting unit 234 is replaced with a contrast. It may be realized by replacing the correction gain setting unit 302.
A case where contrast correction is performed on a target pixel (a pixel to be processed in an image signal) will be described with reference to FIGS. 12 and 13.
FIG. 12 shows a configuration diagram of the contrast correction unit 301.

As illustrated in FIG. 12, the contrast correction unit 301 includes a luminance calculation unit 3000 that calculates luminance from an image signal, and a distribution analysis unit 3001 that acquires a luminance histogram distribution (luminance histogram distribution) calculated by the luminance calculation unit 3000. And a correction adjustment unit 3003 for calculating a correction amount ΔY for correcting the luminance (brightness contrast) based on the luminance histogram distribution acquired by the distribution analysis unit 3001 and the contrast correction gain output from the contrast correction gain setting unit 302. . The contrast correction unit 301 also delays the image signal output from the luminance calculation unit 3000 and adjusts the correction amount ΔY calculated by the correction adjustment unit 3003 to adjust the timing of the luminance correction processing. A luminance correction unit 3004 that performs luminance correction processing on the image signal output from the delay unit 3002.
That is, the contrast correction unit 301 obtains the luminance of the target pixel, obtains a correction amount ΔY for correcting the luminance contrast based on the luminance histogram distribution, and performs contrast correction by adding to the luminance of the target pixel.

  Further, instead of obtaining the luminance histogram distribution by the distribution analysis unit 3001, the size of the peripheral pixel region of the target pixel (the entire image region (the entire pixel region of one image (for example, one frame image) formed from pixels)). Based on two of the luminance low-pass filter value (blurred pixel value) indicating the general luminance distribution in the region having a size of about 1/6 to 1/3 of the above and the luminance value of the target pixel, The correction adjustment unit 3003 may obtain the contrast correction amount ΔY. In this case, the contrast correction gain in the contrast correction unit 301 is determined by, for example, the contrast correction gain setting unit 302 having an LUT (LUT that outputs the contrast correction gain from the depth data and the depth degree) that realizes the characteristics shown in FIG. To be acquired. Then, a correction amount adjustment unit 3003 calculates a correction amount ΔY for performing a contrast correction process on the image signal from the acquired contrast correction gain. The image signal (luminance of the target pixel) output from the delay unit 3002 is corrected by the luminance correction unit 3004 by adding the contrast correction amount ΔY calculated by the correction adjustment unit 3003. .

As shown in FIG. 15A, the contrast enhancement process is performed on an area in which the depth data corresponds to the foreground in the image, and the contrast suppression process is performed on the area corresponding to the background in the depth data in the image. To be implemented.
FIG. 15A shows an example in which contrast suppression can be performed by changing the contrast correction addition amount ΔY from positive to negative in the background where the depth data Z is larger than Z1, but it is necessary to be limited to this. For example, foreground processing when the target pixel is explicitly determined to be included in the foreground area by depth data Z, and for the foreground when it is determined that the target pixel is included in the foreground area The contrast enhancement process or the contrast suppression process may be performed by dividing the process and changing the process performed on the target pixel.
As shown in FIG. 15B, the slope of the depth data-contrast correction gain characteristic curve at the depth data Z = Z0 monotonously decreases as the depth level indicated by the interface or the like increases from “weak” to “medium”. To do. That is, as the depth degree increases from “weak” to “medium”, the depth data-contrast correction gain characteristic curve changes from the curve M to the curve N and further to the curve O as shown in FIG. .

As shown in FIG. 15C, the depth data-contrast correction gain characteristic curve changes from the curve M to the curve N and further to the curve O as the depth degree increases from “weak” to “medium”. The value of the contrast correction gain in the depth data Z = Z1 monotonously decreases from positive to negative. FIG. 15C shows characteristics when the depth is “medium” as a default value, and when the depth is “medium”, the value of the contrast correction gain at depth data Z = Z1. Has a characteristic of “0”.
In the image processing apparatus according to the first modification, the depth effect in the processed image can be enhanced by changing the depth data-contrast correction gain characteristic in this way. That is, in the image processing apparatus according to the modified example 1, the depth effect is realized in the processed image by gradually changing the degree of contrast enhancement according to the depth data Z when the depth level is “weak” to “medium”. As the degree of depth changes from “medium” to “strong”, the contrast enhancement in the foreground area on the image is further strengthened, and the contrast feeling in the foreground area on the image is suppressed, thereby further enhancing the depth effect. Can be controlled to increase.

In the image processing apparatus according to the modified example 1, the contrast suppression process is performed by setting the depth of the depth data Z = Z1 and the contrast correction gain to a negative value in FIGS. 15A and 15C. The contrast correction gain value may be converged to a predetermined minimum gain value with depth data Z = Z1, or may be set to “0”.
However, when the depth degree is “strong”, in order to further enhance the depth effect in the processed image, it is preferable to have the characteristics as shown in FIG. This is because, in the image processing apparatus according to the first modification, the depth effect is further enhanced by controlling with the characteristics as shown in FIG.
(Modification 2)
Next, a second modification of the image processing apparatus 100 according to the present embodiment will be described.

Modification 2 is characterized in that a shadow correction process is performed on an image signal. Since other than that is the same as that of the above-mentioned embodiment, description is abbreviate | omitted.
Note that the image processing apparatus according to the modified example 2 is realized by replacing the depth feeling correction unit 230 with a depth feeling correction unit 2303 illustrated in FIG. 17 in the image processing apparatus 100 of the first embodiment.
The image processing apparatus according to the second modification may be realized by replacing the depth feeling correction unit 230 with the depth feeling correction unit 2304 illustrated in FIG. 18 in the image processing apparatus 100 according to the first embodiment.
Furthermore, in the image processing apparatus according to the first modification, in the image processing apparatus 100 according to the first embodiment, the contour correction unit 231 of the depth sensation correction unit 230 is replaced with a shadow correction unit 303, and the contour correction gain setting unit 234 is replaced with a shadow. Alternatively, the correction gain setting unit 304 may be replaced.

A case where the shadow correction of the target pixel is performed will be described with reference to FIGS. 16 and 17.
FIG. 16 shows a configuration diagram of the shadow correction unit 303.
As illustrated in FIG. 16, the shadow correction unit 303 includes a luminance calculation unit 3000 that calculates luminance from an image signal, a shadow analysis unit 4001 that performs shadow analysis from the luminance calculated by the luminance calculation unit 3000, and a shadow analysis unit 4001. And a shadow addition adjustment unit 4003 that calculates a correction amount ΔY for correcting the luminance of the image signal as a negative value based on the shadow analysis result in FIG. 5 and the shadow correction gain output from the shadow correction gain setting unit 304. Also, the shadow correction unit 303 is based on the delay unit 4002 that delays the image signal output from the luminance calculation unit 3000 and the correction amount ΔY calculated by the shadow addition adjustment unit 4003 in order to adjust the timing of the shadow correction process. A shadow correction processing unit 4004 that performs a shadow correction process on the image signal output from the delay unit 4002.

In other words, the process executed by the shadow correction unit 303 is to increase the shadow of the pixels in the image to produce a depth effect. Here, “increasing the shadow” corresponds to obtaining the correction amount ΔY of the luminance Y in the target pixel as a negative value and adding ΔY to the luminance Y.
As shown in FIG. 16, the shadow correction unit 303 first obtains the luminance Y of the target pixel, and performs a shadow analysis on the target pixel based on the luminance. In this case, the presence / absence of a shadow is determined from the general luminance distribution of the peripheral pixel region of the target pixel (region having a size of about 1/6 to 1/3 of the size of the entire image region), and the shadow portion is determined. It may be emphasized. Also, normal vector information indicating the three-dimensional shape of the target pixel is estimated (for example, a three-dimensional shape is assumed based on the x-direction differential value and the y-direction differential value of the target pixel, and the assumed three-dimensional shape is assumed. (Estimate the normal vector of the (three-dimensional solid curved surface)), and can also add a shadow of a desired portion according to the normal vector information (analyzed from the pixel position of the target pixel and the predetermined light source or within the image) It is also possible to control the addition of a shadow to the target pixel from the positional relationship with the light source position and the estimated normal vector).

The shadow addition adjustment unit 4003 performs a shadow correction on the image signal based on the shadow analysis result by the shadow analysis unit 4001 and the shadow correction gain determined by the characteristics shown in FIG. 19 by the shadow correction gain setting unit 304, for example. A correction amount ΔY (<0) is calculated. Then, the shadow correction unit 303 executes the shadow correction process by adding the correction amount ΔY (<0) calculated by the shadow addition adjustment unit 4003 to the luminance Y of the target pixel.
As shown in FIG. 19A, the shadow enhancement process is performed on the area corresponding to the foreground in the depth data in the image, and the shadow suppression process is equivalent to the background in the depth data in the image. What is implemented in the area.
FIG. 19A shows an example in which shadow suppression can be performed by luminance addition by changing the gain of the shadow correction amount ΔY from positive to negative in the background where the depth data Z is greater than Z1. The process performed when Z indicates the foreground and the process performed when the depth data Z indicates the foreground may be explicitly changed.

As shown in FIG. 19B, as the depth level indicated by the interface or the like increases from “weak” to “medium”, the slope of the depth data-shadow correction gain characteristic curve in the depth data Z = Z0 monotonously decreases. To do. That is, as the depth degree increases from “weak” to “medium”, the depth data-shadow correction gain characteristic curve changes from the curve P to the curve Q and further to the curve R as shown in FIG. . In the image processing apparatus according to the modified example 2, the shadow correction gain is determined based on such a depth data-shadow correction gain characteristic curve, and the shadow correction processing is performed, so that the difference in the shadow correction with respect to the depth data Z is clearer. Can be.
In the image processing apparatus according to the modified example 2, as shown in FIG. 19C, the value of the shadow correction gain in the depth data Z = Z1 is changed from “weak” to “medium” (from the curve P to the curve Q). By setting it to “0” (while changing up to), control is performed so that shadow correction is not performed in the background area from Z = Z1. On the other hand, in the image processing apparatus according to the modified example 2, as the depth degree is changed from “medium” to “strong”, the value of the shadow correction gain in the depth data Z = Z1 is monotonously decreased from “0”. The shadow is suppressed rather than the shadow enhancement.

In the image processing apparatus according to the modified example 2, the depth feeling effect can be enhanced in the processed image by changing the depth data-shadow correction gain characteristic in this way. That is, in the image processing apparatus according to the modified example 2, when the depth level is “weak” to “medium”, the depth enhancement effect is realized by gradually changing the degree of shadow emphasis according to the depth data Z, and the depth effect is achieved. As the degree increases from “medium” to “strong”, the shadow enhancement in the foreground area is further enhanced, and the shadow effect is suppressed to suppress the stereoscopic effect in the foreground area, thereby further enhancing the depth effect. Can do.
In FIG. 19A and FIG. 19C, the shadow suppression processing is performed by setting the shadow correction gain value to a negative value (correction amount ΔY is a positive value) with depth data Z = Z1. The value of the shadow correction gain may be converged to a predetermined minimum gain value with the depth data Z = Z1, or may be “0”.

In the image processing apparatus 100 of the present embodiment, it has been described that only the processing for foreground is performed until the depth level reaches a predetermined value. Until then, only distant views may be processed.
Further, in the image processing apparatus 100, image processing that is paired with the near-field region and the far-field region on the image, such as contour correction and blur correction, saturation enhancement and saturation suppression, is executed in various combinations. In addition, the image processing adjustment may be performed by adjusting the degree of depth using a one-dimensional scale as described in the present embodiment.
In the image processing apparatus according to the present embodiment, the case where the depth degree can be adjusted has been described. However, the present invention is not limited to this. You may make it perform by control.

In the image processing apparatus according to the present embodiment, the user can clearly adjust the depth level, but the present invention is not limited to this, and a display mode used when displaying a video on a thin TV or the like. (For example, a cinema mode that emphasizes black gradation and color reproduction so that it is suitable for cinema display, and dynamic that is bright and sharp enough to be suitable for display in a room or store with strong ambient light. For example, a standard display mode for performing normal video display) and the like may be determined.
[Other Embodiments]
Although the present invention has been described based on the above-described embodiments and modifications, it is needless to say that the present invention is not limited to the above-described embodiments. The following cases are also included in the present invention.
(1) Each of the above devices is specifically a computer system including a microprocessor, a ROM, a RAM, and the like. A computer program is stored in the RAM. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

(2) A part or all of the constituent elements constituting each of the above-described devices may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.
(3) Part or all of the constituent elements constituting each of the above devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

(4) The present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.
The present invention also provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). ), Recorded in a semiconductor memory or the like. The digital signal may be recorded on these recording media. In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like. The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like and executed by another independent computer system. You may do that.
(5) The above embodiment and the above modifications may be combined.
Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware. Needless to say, when the image processing apparatus according to the above-described embodiment is realized by hardware, it is necessary to adjust timing for performing each process. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.

  The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  The image processing apparatus, the image processing method, the program, the recording medium, and the integrated circuit according to the present invention have a feature that the user can specify the degree of depth, and are useful for image processing of a display system such as a TV. It can also be applied to the use of emphasizing the sense of depth as post-processing of movies taken by individuals, DVDs, and videos.

1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention. Block diagram of conventional depth emphasis device Block diagram of depth sense correction unit Schematic diagram of adjustment GUI for depth Block diagram of contour correction unit Block diagram of the saturation correction unit (A) Graph showing the relationship between depth data and contour correction gain (b) Graph showing the relationship between the depth degree and the slope of the depth data-contour correction gain characteristic curve (straight line) (A) Graph showing the relationship between depth data and blur correction gain parameter (b) Graph showing the relationship between the depth degree and the slope of the depth data-blur correction gain parameter characteristic curve (straight line) (A) Graph showing the relationship between depth data and saturation correction gain (b) Graph showing the relationship between depth and depth data-saturation correction gain characteristic curve (straight line) (c) Depth and saturation The graph which shows the relationship of the value in z1 of a correction gain The figure which shows the relationship between the depth degree and the contour correction gain and blur correction gain The figure which shows the relationship between the depth degree and the gain for saturation correction and the gain for contour correction The block diagram of the contrast correction part which concerns on the modification 1 of 1st Embodiment. The block diagram of the depth feeling correction | amendment part which concerns on the modification 1 of 1st Embodiment. The block diagram of the depth feeling correction | amendment part which concerns on the modification 1 of 1st Embodiment. The figure which shows the relationship between depth data and contrast correction gain The block diagram of the shadow correction part which concerns on the modification 2 of 1st Embodiment. The block diagram of the depth feeling correction | amendment part which concerns on the modification 2 of 1st Embodiment. The block diagram of the depth feeling correction | amendment part which concerns on the modification 2 of 1st Embodiment. Diagram showing the relationship between depth data and shading correction gain

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 200 Image data input part 210 Depth data input part 220 Depth degree input part 230, 2301, 2302, 2303, 2304 Depth feeling correction part 231 Contour correction part 2311 Delay part 2312 High pass part 2313 Gain adjustment part 2314 Addition part 232 Blur correction unit 233 Saturation correction unit 2331 HSV conversion unit 2332 Delay unit 2333 Saturation addition unit 2334 Inverse HSV conversion unit 234 Contour correction gain setting unit 235 Blur correction gain setting unit 236 Saturation correction gain setting unit 240 Image data output unit 301 Contrast correction unit 302 Contrast correction gain setting unit 303 Shadow correction unit 304 Shadow correction gain setting unit

Claims (30)

An image data input unit to which an image data signal capable of forming an image composed of pixels is input;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired A depth data input unit to which depth data correlated with the distance is input;
A depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
A depth sensation correction unit that performs a depth sensation correction on the image data signal based on the one-dimensional signal, and when the one-dimensional signal is equal to or less than a predetermined value, A depth sensation correction unit that performs only the correction process and, when the one-dimensional signal is larger than a predetermined value, performs the near-field correction process and the far-field correction process on the image data signal ;
An image processing apparatus comprising: The depth correction unit
A contour correction gain setting unit that sets a contour correction gain based on the depth data and the one-dimensional signal;
A contour correction unit that performs contour correction on the image data signal based on the contour correction gain;
A blur correction gain setting unit for setting a blur correction parameter based on the depth data and the one-dimensional signal;
A blur correction unit for performing blur correction on the image data signal based on the blur correction parameter;
Based on the depth data and the one-dimensional signal, a selection unit that selects one of the image data signal whose contour has been corrected by the contour correction unit and the image data signal whose blur has been corrected by the blur correction unit When,
Having
The image processing apparatus according to claim 1. The contour correction gain setting unit increases the value of the one-dimensional signal, the contour correction gain for the foreground region on the image with the smaller depth data value, and the distant view on the image with the larger depth data value. Setting the contour correction gain so that a difference between the contour correction gain and the region is large;
The image processing apparatus according to claim 2. The blur correction gain setting unit sets the blur correction parameter so that blur processing is not executed regardless of the value of the one-dimensional signal during the depth data from Z0 to Z2 (> Z0). When the depth data exceeds Z2, as the value of the one-dimensional signal increases, the degree of blur processing when the depth data value is Z2, and the value of the depth data is Z1 (> Z2). Set the blur correction parameter so that the difference from the degree of blur processing at a certain time becomes large.
The image processing apparatus according to claim 2. An image data input unit to which an image data signal capable of forming an image composed of pixels is input;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired A depth data input unit to which depth data correlated with the distance is input;
A depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
A depth sensation correction unit that performs depth sensation correction on the image data signal based on the one-dimensional signal;
With
The depth correction unit
A contour correction gain setting unit that sets a contour correction gain based on the depth data and the one-dimensional signal;
A contour correction unit that performs contour correction on the image data signal based on the contour correction gain;
A blur correction gain setting unit for setting a blur correction parameter based on the depth data and the one-dimensional signal;
A blur correction unit for performing blur correction on the image data signal based on the blur correction parameter;
Based on the depth data and the one-dimensional signal, a selection unit that selects one of the image data signal whose contour has been corrected by the contour correction unit and the image data signal whose blur has been corrected by the blur correction unit When,
Have
When the value of the one-dimensional signal is equal to or less than the first value ZT, the selection unit selects the image data signal whose contour is corrected by the contour correction unit, and the value of the one-dimensional signal is the first value ZT. If the depth data value is less than or equal to the first depth data value Z2, the image data signal whose contour has been corrected by the contour correction unit is selected, and the value of the one-dimensional signal is the first value. If the value of the depth data is greater than the first depth data value Z2, the image data signal that has been subjected to blur correction by the blur correction unit is selected.
Image processing device. The depth correction unit
A saturation correction gain setting unit that sets a saturation correction gain based on the depth data and the one-dimensional signal;
A saturation correction unit that performs saturation correction on the image data signal based on the saturation correction gain;
The image processing apparatus according to claim 2. The saturation correction gain setting unit increases the saturation correction gain for the foreground region on the image where the depth data value is small and the depth data value is large as the value of the one-dimensional signal is large. The saturation correction gain is set so that a difference between the saturation correction gain and the distant view area increases.
The image processing apparatus according to claim 6. The depth correction unit
A contrast correction gain setting unit that sets a contrast correction gain based on the depth data and the one-dimensional signal;
A contrast correction unit configured to perform contrast correction on the image data signal based on the contrast correction gain;
The image processing apparatus according to claim 1. The contrast correction gain setting unit increases a contrast correction gain for a foreground area on the image having a smaller depth data value and a larger background value on the image having a larger depth data value as the value of the one-dimensional signal increases. Setting the contrast correction gain so that a difference between the contrast correction gain and the region is large;
The image processing apparatus according to claim 8. The depth correction unit
A shadow correction gain setting unit that sets a shadow correction gain based on the depth data and the one-dimensional signal;
A shadow correction unit that performs shadow correction on the image data signal based on the shadow correction gain;
The image processing apparatus according to claim 1. The shadow correction gain setting unit increases the shadow correction gain for the foreground region on the image where the depth data value is small and the depth data value is large as the value of the one-dimensional signal is large. Setting the shading correction gain so that the difference between the shading correction gain and the region is large;
The image processing apparatus according to claim 10 . The depth correction unit corresponds to the correction amount for the image data signal corresponding to the pixel of the nearest scene on the image and the pixel of the farthest view on the image as the value of the one-dimensional signal increases. Expanding the difference with the correction amount for the image data signal
The image processing apparatus according to claim 1. The depth sensation correction unit corresponds to the correction amount for the image data signal corresponding to the pixel of the nearest scene on the image and the pixel of the farthest view on the image regardless of the value of the one-dimensional signal. The difference between the correction amount for the image data signal is made constant,
The image processing apparatus according to claim 1. The depth sensation correction unit performs a contour enhancement process as a foreground correction process for the image data signal, and performs a blur correction process as a distant view correction process for the image data signal.
The image processing apparatus according to claim 1 . The depth sensation correction unit performs saturation enhancement processing as a foreground correction processing for the image data signal, and performs saturation dulling processing as a background correction processing for the image data signal.
The image processing apparatus according to claim 1 . The depth sensation correction unit performs a correction process on a predetermined feature amount of the image based on the one-dimensional signal.
The image processing apparatus according to claim 1. The depth sensation correction unit performs a correction process on the image data signal that forms an image including a first area and a second area that can be distinguished by the depth data.
The correction process is performed on the image data signal in the first area by controlling the degree of enhancement of the correction process, and the correction process is performed on the image data signal in the second area. Performing the correction process by controlling the degree of suppression of
The image processing apparatus according to claim 16 . The depth sensation correction unit performs the correction process on the image data signal in the first region based on the one-dimensional signal by controlling the enhancement degree of the correction process. For the image data signal in the region, the correction process is performed by controlling the degree of suppression of the correction process.
The image processing apparatus according to claim 17 . The depth sensation correction unit performs contour emphasis processing on the image data signal in the first area, and performs blur correction processing on the image data signal in the second area. Do,
The image processing apparatus according to claim 17 . The depth correction unit performs a saturation enhancement process on the image data signal in the first area, and performs a saturation reduction process on the image data signal in the second area. ,
The image processing apparatus according to claim 17 . The depth sensation correction unit performs contrast enhancement processing on the image data signal in the first region and performs contrast suppression processing on the image data signal in the second region.
The image processing apparatus according to claim 17 . The depth sensation correction unit performs a shadow enhancement process on the image data signal in the first area, and performs a shadow suppression process on the image data signal in the second area.
The image processing apparatus according to claim 17 . The depth sensation correction unit performs a correction process on the image data signal that forms an image including a first area and a second area that can be distinguished by the depth data.
A first correction process is performed on the image data signal in the first area, and a second correction process different from the first correction process is performed on the image data signal in the second area. Do,
The image processing apparatus according to claim 16 . The depth sensation correction unit selects any one of four correction processes of a contour correction process, a saturation correction process, a contrast correction process, and a shadow correction process as the first correction process, and the selected first correction Processing, and as the second correction process, select from three correction processes except the one selected as the first correction process from the four correction processes, and perform the selected second correction process.
The image processing apparatus according to claim 23 . The depth correction unit controls the intensity of the first correction process and the intensity of the second correction process based on the one-dimensional signal;
The image processing apparatus according to claim 23 . An image data input unit to which an image data signal capable of forming an image composed of pixels is input;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired A depth data input unit to which depth data correlated with the distance is input;
A depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
When the value of the one-dimensional signal is equal to or less than a predetermined value, edge enhancement processing is performed on the image data signal in the foreground area on the image, and when the value of the one-dimensional signal is larger than a predetermined value, A depth correction unit that performs an edge enhancement process on the image data signal in the foreground region on the image, and performs a blur correction process on the image data signal in the distant region on the image;
An image processing apparatus having
A controller having an operation unit capable of inputting the one-dimensional signal to the depth degree input unit of the image processing apparatus by a one-dimensional operation by a user;
An image processing system comprising: An image data input step for inputting an image data signal capable of forming an image composed of pixels;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired A depth data input step for inputting depth data correlated with the distance ;
A depth degree input step for inputting a one-dimensional signal for controlling a sense of depth in the image formed by the image data signal;
A depth correction step for correcting a depth sensation on the image data signal based on the one-dimensional signal, and when the one-dimensional signal is a predetermined value or less, Depth sensation correcting step for performing only the correction process, and when the one-dimensional signal is larger than a predetermined value, performing the near-field correction process and the far-field correction process on the image data signal ;
An image processing method comprising: An image data input unit to which an image data signal that can form an image composed of pixels is input to a computer;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired Depth data input part to which depth data correlated with distance is input,
Depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
A depth sensation correction unit that performs a depth sensation correction on the image data signal based on the one-dimensional signal, and when the one-dimensional signal is equal to or less than a predetermined value, Depth sensation correction unit that performs only correction processing and, when the one-dimensional signal is greater than a predetermined value, performs the near-field correction processing and the far-field correction processing on the image data signal ,
Program to function as. An image data input unit to which an image data signal that can form an image composed of pixels is input to a computer;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired Depth data input part to which depth data correlated with distance is input,
Depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
A depth sensation correction unit that performs a depth sensation correction on the image data signal based on the one-dimensional signal, and when the one-dimensional signal is equal to or less than a predetermined value, Depth sensation correction unit that performs only correction processing and, when the one-dimensional signal is greater than a predetermined value, performs the near-field correction processing and the far-field correction processing on the image data signal ,
A computer-readable recording medium on which a program for functioning as a computer is recorded. An image data input unit to which an image data signal capable of forming an image composed of pixels is input;
A position in an actual three-dimensional space corresponding to a pixel or a pixel block on a two-dimensional image formed by the image data signal and a position in the three-dimensional space of the imaging device when the two-dimensional image is acquired A depth data input unit to which depth data correlated with the distance is input;
A depth degree input unit to which a one-dimensional signal for controlling a sense of depth is input in the image formed by the image data signal;
A depth sensation correction unit that performs a depth sensation correction on the image data signal based on the one-dimensional signal, and when the one-dimensional signal is equal to or less than a predetermined value, A depth sensation correction unit that performs only the correction process and, when the one-dimensional signal is larger than a predetermined value, performs the near-field correction process and the far-field correction process on the image data signal ;
An integrated circuit comprising:

Images