JP2014191430A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of performing image quality processing to ensure a sensation of depth.SOLUTION: The image processing system includes: a feature amount detection section 10; a depth information acquisition section 20; a determination section 30; and a processing section 40. The feature amount detection section 10 detects a feature amount of each pixel included in an input image. The depth information acquisition section 20 acquires a depth amount which represents a position of each pixel included in the input image in a depth direction. The determination section 30 determines the blur level of each pixel included in the input image based on a difference in the depth amount with respect to a depth amount of a pixel which has the largest feature amount. The processing section 40 performs an image quality processing on each pixel included in the input image based on the blur level, which is different from each other.

Description

  The present disclosure relates to an image processing apparatus and an image processing method.

  Patent Document 1 detects a depth value for each pixel of an input image and specifies a composition based on the depth value. Then, based on the composition information, the input image is subjected to high image quality processing.

JP 2010-72982 A

  In the present disclosure, the degree of blur of each pixel of the input image is determined based on the difference in depth from the depth of the pixel having the largest feature amount among the pixels of the input image, and the pixel of the input image is determined based on the determined degree of blur. An image processing apparatus and an image processing method capable of performing image quality processing that reduces the loss of depth are provided because different image quality processing is performed for each image.

  The image processing apparatus according to the present disclosure includes a feature amount detection unit that detects a feature amount of each pixel of the input image, a depth information acquisition unit that acquires a depth amount representing a position in the depth direction of each pixel of the input image, A determining unit that determines the degree of blur of each pixel of the input image based on a difference in depth from the depth amount of the pixel having the largest feature amount, and an image quality that differs for each pixel of the input image based on the degree of blur A processing unit that performs processing.

  Further, the image processing method of the present disclosure includes a feature amount detection step for detecting a feature amount of each pixel of the input image, and a depth information acquisition step for acquiring a depth amount representing a position in the depth direction of each pixel of the input image. Determining the degree of blur of each pixel of the input image based on the difference in depth from the depth of the pixel having the largest feature amount; and for each pixel of the input image based on the degree of blur Processing steps for performing different image quality processing.

  According to the image processing device and the image processing method of the present disclosure, the degree of blur of each pixel of the input image is determined based on the difference in depth amount from the depth amount of the pixel having the largest feature amount among the pixels of the input image, Since different image quality processing is performed for each pixel of the input image based on the determined degree of blur, it is possible to perform image quality processing that reduces the loss of depth.

  According to the image processing device and the image processing method of the present disclosure, the degree of blur of each pixel of the input image is determined based on the difference in depth amount from the depth amount of the pixel having the largest feature amount among the pixels of the input image, Since different image quality processing is performed for each pixel of the input image based on the determined degree of blur, it is possible to perform image quality processing that reduces the loss of depth.

2 is a block diagram illustrating a configuration of an image processing apparatus according to Embodiment 1. FIG. 5 is a flowchart for describing image processing of the image processing apparatus according to Embodiment 1; Explanatory drawing of the feature-value detection process of the feature-value detection part in Embodiment 1. FIG. Explanatory drawing of the blur degree determination process of the determination part in Embodiment 1. FIG. Explanatory drawing of the blur degree determination process of the determination part in Embodiment 1. FIG. Explanatory drawing of the blur degree determination process of the determination part in Embodiment 1. FIG. FIG. 3 is an explanatory diagram of image quality processing of a processing unit in the first embodiment. Explanatory drawing of the depth boundary pixel specific process of the depth information acquisition part in Embodiment 1. FIG. FIG. 4 is a block diagram illustrating a configuration of an image processing apparatus according to Embodiment 2. Explanatory drawing of the depth information correction process of the depth information correction part in Embodiment 2. FIG. The block diagram which shows the structure of the image processing apparatus in other embodiment. Explanatory drawing of the area | region specific process of the area | region specific part in other embodiment. The block diagram which shows the structure of the image processing apparatus in other embodiment.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

The inventor (s) provides the accompanying drawings and the following description in order for those skilled in the art to fully understand the present disclosure, and is intended to limit the subject matter described in the claims. Not what you want.
(Embodiment 1)
[1-1. Constitution]
FIG. 1 is a block diagram illustrating a configuration of the image processing apparatus according to the first embodiment.

  In FIG. 1, the image processing apparatus 1 includes a feature amount detection unit 10, a depth information acquisition unit 20, a determination unit 30, and a processing unit 40.

  The feature amount detection unit 10 detects the feature amount of each pixel of the input image and outputs it to the determination unit 30.

  The depth information acquisition unit 20 acquires a depth amount representing the position in the depth direction of each pixel of the input image, and outputs the acquired depth amount to the determination unit 30.

  The depth information acquisition unit 20 may calculate the depth amount of each pixel based on the input image, or when the depth amount of each pixel is given in advance, the depth amount given as it is may be used.

  The determination unit 30 receives the feature amount of each pixel of the input image detected by the feature amount detection unit 10 and the depth amount of each pixel of the input image acquired by the depth information acquisition unit 20. The determination unit 30 specifies the pixel having the largest feature amount from the pixels of the input image. The determination unit 30 obtains the difference between the depth amount of the pixel having the largest feature amount and the depth amount of the other pixels of the input image, determines the degree of blur of each pixel of the input image based on the difference, and Output. That is, the determination unit 30 determines the degree of blur of each pixel of the input image based on the depth amount difference from the depth amount of the pixel having the largest feature amount, and outputs the determined degree to the processing unit 40.

  The processing unit 40 performs different image quality processing for each pixel of the input image based on the degree of blur of each pixel of the input image determined by the determination unit 30.

[1-2. Operation]
The operation of the image processing apparatus 1 configured as described above will be described below.

  FIG. 2 is a flowchart illustrating image processing of the image processing apparatus according to the first embodiment.

[Feature detection processing]
The feature amount detection unit 10 detects the feature amount of each pixel of the input image and outputs it to the determination unit 30 (step S21).

  FIG. 3 is an explanatory diagram of the feature amount detection processing of the feature amount detection unit according to the first embodiment.

  FIG. 3A is a block diagram illustrating the configuration of the Sobel filter coefficient (vertical and horizontal) and the feature amount detection unit 10 for detecting the feature amount of each pixel of the input image. FIG. 3B is a block diagram showing another configuration of the Laplacian filter coefficient and feature quantity detection unit 10 for detecting the feature quantity of each pixel of the input image.

  The feature amount detection unit 10 detects the feature amount of each pixel of the input image by performing, for example, the filtering process illustrated in FIG. 3 on each pixel of the input image. As the feature value of each pixel of the input image, for example, a primary differential value (FIG. 3A) or a secondary differential value (FIG. 3B) in the luminance component of the input image is raised. Hereinafter, the feature amount detection processing of each pixel of the input image in each case will be described.

  In FIG. 3A, the primary differential filter (vertical) 11 and the primary differential filter (horizontal) 12 apply the Sobel filter shown in FIG. 3A to the pixel of interest and the surrounding 8 pixels of the input image. A differential value dx in the horizontal direction and a differential value dy in the vertical direction are calculated. The primary differential value calculation unit 13 obtains the feature amount of the pixel of interest output by the square root of the square from the differential value dx calculated by the primary differential filter (vertical) 11 and the differential value dy calculated by the primary differential filter (horizontal) 12. . Further, the primary differential value calculation unit 13 may simply obtain the feature amount of the target pixel to be output from the sum of the absolute values of the differential values dx and dy.

  Next, in FIG. 3B, the secondary differential filter 14 applies the Laplacian filter shown in FIG. 3B to the pixel of interest and the surrounding 8 pixels of the input image, thereby obtaining the secondary differential value of the pixel of interest. Is calculated. Thereafter, the absolute value calculation unit 15 calculates the absolute value of the secondary differential value as the feature amount of the pixel of interest.

[Depth information acquisition processing]
The depth information acquisition unit 20 acquires a depth amount representing the position in the depth direction of each pixel of the input image, and outputs the acquired depth amount to the determination unit 30 (step S22).

  The depth information acquisition unit 20 acquires the depth amount of each pixel of the input image. For example, when the input image is a three-dimensional image having a plurality of viewpoint images, the depth amount of the pixels of the input image is calculated by block matching between the left-eye frame and the right-eye frame. . When the input image is a two-dimensional image, the depth amount of each pixel of the input image is calculated in a pseudo manner from composition analysis of the input image, luminance information, and the like.

  The depth information acquisition process by the depth information acquisition unit 20 is not necessarily performed after the feature amount detection process by the feature amount detection unit 10, and may be performed before or simultaneously with the feature amount detection process by the feature amount detection unit 10. good.

  The depth information acquisition unit 20 may calculate the depth amount of each pixel based on the input image as described above. When the depth amount of each pixel is given in advance, the depth amount given as it is is used. May be.

  In the present embodiment, the depth amount of each pixel of the input image will be described below assuming that the value increases as the distance from the viewer increases.

[Bokeh degree determination process]
The determination unit 30 receives the feature amount of each pixel of the input image detected by the feature amount detection unit 10 and the depth amount of each pixel of the input image acquired by the depth information acquisition unit 20. The determination unit 30 specifies the pixel having the largest feature amount from the pixels of the input image. The determination unit 30 obtains the difference between the depth amount of the pixel having the largest feature amount and the depth amount of the other pixels of the input image, determines the degree of blur of each pixel of the input image based on the difference, and Output. That is, the determination unit 30 determines the degree of blur of each pixel of the input image based on the depth amount difference from the depth amount of the pixel having the largest feature amount, and outputs it to the processing unit 40 (step S23).

  4 to 6 are explanatory diagrams of the blur degree determination process of the determination unit according to the first embodiment.

  In FIG. 4, the determination unit 30 includes a feature amount distribution calculation unit 31, a feature amount distribution normalization unit 32, and a blur degree calculation unit 33.

  The feature amount distribution calculation unit 31 receives the feature amount of each pixel of the input image detected by the feature amount detection unit 10 and the depth amount of each pixel of the input image acquired by the depth information acquisition unit 20. A distribution of the feature amount with respect to the depth amount of the pixel is generated and output to the feature amount distribution normalization unit 32.

  Hereinafter, the feature amount distribution calculation process for the depth amount of each pixel of the input image by the feature amount distribution calculation unit 31 will be described with reference to FIG.

  In FIG. 5A, each pixel of the input image (here, all 25 pixels) is classified into five levels (0 to 50, 51 to 100, 101 to 150, 151 to 200, 201 to 255) according to the depth amount. This is an example.

  For example, five pixels having values of 0 to 50 as depth amounts have 10, 5, 7, 9, 4 as feature amounts, respectively.

  As shown in FIG. 5A, the feature amount distribution calculation unit 31 classifies all 25 pixels of the input image into five levels according to the depth amount of each pixel.

  FIG. 5B shows an example in which one representative feature amount is set at each depth amount step from the pixel feature amounts classified into five steps according to the depth amount.

  As a method for setting the representative value of the feature value at each depth level, for example, the following method can be used.

  One method for setting the representative value of the feature amount at each depth amount stage is a method of calculating the average value of the feature amounts of the pixels belonging to each depth amount stage. For example, the representative value 7 of the feature quantity is calculated as an average value of the feature quantities (10, 5, 6, 9, 4) of five pixels having a value of 0 to 50 as the depth quantity.

  Another method of setting the representative value of the feature amount at each depth amount stage is a method in which coring is used in combination with the above-described method for calculating the average value of the feature amounts of pixels belonging to each depth amount stage. This is a method of calculating an average value of feature amounts of pixels belonging to each depth amount stage, assuming that there is no pixel having a feature amount less than a certain value. For example, assuming that a constant value is 1 and a feature amount of 5 pixels having a value of 51 to 100 as a depth amount is considered to be a pixel having a feature amount of 0, 4 pixels having a value of 51 to 100 as a depth amount As a mean value of the feature amounts (13, 14, 7, 9), a representative value 10.75 of the feature amount is calculated.

  Another method of setting the representative value of the feature amount at each depth amount stage is a method of calculating the maximum value of the feature amount of the pixels belonging to each depth amount stage. For example, the representative value 21 of the feature amount is calculated as the maximum value of the feature amounts (21, 18, 20, 0, 14) of five pixels having values of 101 to 150 as the depth amount. At this time, in order to eliminate the influence of noise, the top few percent of the feature amounts of the pixels belonging to each depth amount stage may be excluded, and the maximum value of the feature amount of the remaining pixels may be used as the representative value.

  As shown in FIG. 5B, the feature amount distribution calculation unit 31 calculates the representative value of the feature amount of the pixel belonging to each depth amount stage.

  FIG. 5C is an example in which the relationship between the depth amount and the feature amount of each pixel of the input image is graphed. Here, the x mark shown in FIG. 5C indicates the representative value of the pixel belonging to each depth amount stage, and the depth amount and the feature amount of each pixel of the input image are based on the x mark. Graph the relationship. At this time, the graph indicating the relationship between the depth amount and the feature amount of each pixel of the input image may be set so as to pass through all the x marks, or may be approximated to a quadratic function using a least square method or the like. .

  As shown in FIG. 5C, the feature amount distribution calculation unit 31 calculates the representative value of the feature amount of the pixel belonging to each depth amount stage, and then the relationship between the depth amount of each pixel of the input image and the feature amount. Is graphed.

  The feature amount distribution normalization unit 32 receives the feature amount distribution with respect to the depth amount of each pixel of the input image from the feature amount distribution calculation unit 31, and the feature amount of the pixel having the largest feature amount among the pixels of the input image is limited to the upper limit. The feature amount distribution is normalized so as to be output to the blur degree calculation unit 33.

  The blur degree calculation unit 33 receives the distribution of the feature amount of each pixel of the input image normalized from the feature amount distribution normalization unit 32, and determines the blur degree of each pixel. Specifically, the blur degree calculation unit 33 is based on the distribution of the feature amount of each pixel of the normalized input image, and the depth amount of the pixel having the lowest feature amount and the largest feature amount of the pixel having the largest feature amount. The degree of blur is determined to be higher as the pixel has a larger difference in depth.

  Hereinafter, with reference to FIG. 6, the feature amount distribution normalization unit 32 normalizes the distribution of the feature amount with respect to the depth amount of each pixel of the input image and the blur degree calculation unit 33 normalizes each pixel of the input image. A blur degree calculation process for each pixel of the input image based on the distribution of the feature amount with respect to the depth amount will be described.

  FIG. 6A is an example in which the distribution of the feature amount with respect to the depth amount of each pixel of the input image is normalized. FIG. 6B is an example in which the degree of blur of each pixel of the input image is calculated based on the distribution of the feature amount with respect to the depth amount of each pixel of the normalized input image.

  The feature amount distribution normalization unit 32 normalizes the entire feature amount distribution graph so that the feature amount of the pixel having the largest feature amount among the pixels of the input image becomes the upper limit. For example, the feature amount distribution normalization unit 32, when the value range of the feature amount of the pixel of the input image is 0 to 255 and the maximum feature amount of the pixel of the input image is 150, the feature amount with respect to the depth amount of each pixel of the input image Is normalized so that the feature value of the distribution graph of the feature value with respect to the depth value of each pixel of the input image takes a value in the range of 0 to 255. Do.

  Next, the blur degree calculation unit 33 calculates a feature amount corresponding to the depth amount of each pixel of the input image, and calculates the feature amount of each pixel of the input image from the maximum value that can be taken by the feature amount of each pixel of the input image. The subtracted value is calculated as the degree of blur. For example, assuming that the range of the feature amount of the pixel of the input image is 0 to 255, the depth amount of a pixel of the input image is 50, and the feature amount corresponding to the depth amount is 80, the blur degree calculation unit 33 The degree of blur is 255−80 = 175.

[Image quality processing]
The processing unit 40 performs different image quality processing for each pixel of the input image based on the degree of blur of each pixel of the input image determined by the determination unit 30 (step S24).

  FIG. 7 is an explanatory diagram of the image quality processing of the processing unit in the first embodiment.

  FIG. 7 shows processing in accordance with the degree of blur in each of (1) edge enhancement processing (gain, enhancement band), (2) smoothing processing (gain, smoothing band), and (3) contrast processing (gain). Indicates that the degree of change.

  The processing unit 40 (1) changes two gains (processing amount) and enhancement band in accordance with the degree of blur in the edge enhancement process. Regarding the gain (processing amount) in the edge enhancement process, if the enhancement process is performed on a pixel having a large degree of blur (not in focus), the blur is reduced and the sense of depth is impaired. Therefore, processing is performed so that the gain (processing amount) is reduced as the degree of blur increases. Further, the processing unit 40 has a fine pattern (high frequency component) in a pixel with a low degree of blur (focused) in the enhanced frequency band in the edge enhancement process, but has a high degree of blur (high focus). Pixels that are not matched do not inherently have fine patterns, so edge enhancement processing is performed in the high frequency region as the degree of blur is smaller. As described above, the processing unit 40 can appropriately perform image quality processing (edge enhancement processing) without impairing the sense of depth of the input image.

  Further, the processing unit 40 changes two of the gain (processing amount) and the smoothing band according to the degree of blur in (2) smoothing processing (including noise removal processing). When the gain (processing amount) in the smoothing process is strongly applied to a pixel with a low degree of blurring (in focus), the processing unit 40 blurs the image and loses a feeling of focus. Therefore, processing is performed so that the gain (processing amount) is decreased as the degree of blur is reduced. Further, since the processing unit 40 smoothes the smoothed frequency band in the smoothing process to a low frequency band on a pixel with a low degree of blur (in focus), the focus feeling is lost. Smoothing processing is performed at a higher frequency for pixels with a lower degree of blur. As described above, the processing unit 40 can appropriately perform image quality processing (smoothing processing) without impairing the sense of depth of the input image.

  In addition, regarding the (3) contrast processing (gain), the processing unit 40 applies the contrast processing more strongly to a pixel having a smaller degree of blur (in focus). As described above, the processing unit 40 can emphasize a sense of depth by highlighting a pixel of interest (= focused pixel) with a low degree of blur.

[Other processing]
[Depth boundary pixel identification processing]
The depth information acquisition unit 20 acquires a depth amount representing the position in the depth direction of each pixel of the input image and outputs the depth amount to the determination unit 30. In addition, the depth information acquisition unit 20 performs input based on the difference in depth amount between each pixel of the input image. A depth boundary pixel located at the boundary between the foreground and the background of the image may be specified and output to the determination unit 30.

  FIG. 8 is an explanatory diagram of the depth boundary pixel specifying process of the depth information acquisition unit in the first embodiment.

  FIG. 8A shows a case where the input image includes a foreground having a depth amount 20 and a background having a depth amount 60. FIG. 8B shows the pixels located in the area surrounded by the broken line in FIG. 8A and the depth amount of each pixel.

  The depth information acquisition unit 20 identifies a pixel in which the difference between the target pixel of the input image and the surrounding eight pixels is equal to or greater than a certain level, and outputs the pixel to the determination unit 30 as a depth boundary pixel located at the foreground / background boundary of the input image. To do. In the pixel shown in FIG. 8B, four pixels having a depth amount 30 are specified as depth boundary pixels.

  In addition to the feature amount of each pixel of the input image detected by the feature amount detection unit 10 and the depth amount of each pixel of the input image acquired by the depth information acquisition unit 20, the determination unit 30 receives the depth from the depth information acquisition unit 20. Receive border pixels. The determination unit 30 specifies the pixel having the largest feature amount from the pixels of the input image. The determination unit 30 obtains the difference between the depth amount of the pixel having the largest feature amount and the depth amount of the other pixels of the input image, determines the degree of blur of each pixel of the input image based on the difference, and Output. Here, the determination unit 30 identifies pixels in the area surrounded by the depth boundary pixels specified by the depth information acquisition unit 20, and determines the degree of blur of the depth boundary pixels in the area surrounded by the depth boundary pixels. Higher or lower than the degree of blur of the pixel in the pixel.

  The processing unit 40 performs different image quality processing for each pixel of the input image based on the degree of blur of each pixel of the input image determined by the determination unit 30.

  In the determination unit 30, the pixels in the region surrounded by the depth boundary pixels specified by the depth information acquisition unit 20 are specified, and the degree of blur of the depth boundary pixels is determined by the pixels in the region surrounded by the depth boundary pixels. The processing unit 40 performs different image quality processing for each pixel of the input image based on the degree of blur of each pixel of the input image determined by the determination unit 30. It is possible to adjust the influence on the feeling of depth by controlling the separation. For example, if the edge enhancement gain (processing amount) is increased only for the depth boundary pixels, the processing unit 40 can reduce the sense of depth because objects with different depths appear to be separated from each other. On the other hand, if the edge enhancement gain (processing amount) is increased too much only for the depth boundary pixels, the processing unit 40 feels uncomfortable as if the objects were separated more than necessary and the pictures taken separately were combined (writing Since there is a possibility that a split feeling may occur, it is possible to reduce the edge emphasis gain and reduce the split feeling.

[1-3. Effect]
According to the image processing device and the image processing method of the present disclosure, the degree of blur of each pixel of the input image is determined based on the difference in depth amount from the depth amount of the pixel having the largest feature amount among the pixels of the input image, Since different image quality processing is performed for each pixel of the input image based on the determined degree of blur, it is possible to perform image quality processing that reduces the loss of depth. In addition, the pixels in the area surrounded by the depth boundary pixels and the depth boundary pixels located at the boundary between the foreground and the background of the input image are identified, and the degree of blur of the depth boundary pixels is determined in the area surrounded by the depth boundary pixels. Therefore, it is possible to adjust the influence on the sense of depth by controlling the separation between objects.

In the first embodiment, the feature amount detection unit 10 corresponds to an example of a feature amount detection unit, the depth information acquisition unit 20 corresponds to an example of a depth information acquisition unit, the determination unit 30 corresponds to an example of a determination unit, The processing unit 40 corresponds to an example of a processing unit. The feature amount distribution calculating unit 31 corresponds to an example of a feature amount distribution calculating unit, and the blur degree calculating unit 33 corresponds to an example of a blur degree calculating unit.
(Embodiment 2)
[2-1. Constitution]
FIG. 9 is a block diagram illustrating a configuration of the image processing apparatus according to the second embodiment.

  A difference from the image processing apparatus 1 of FIG. 1 is that a depth information correction unit 50 is further provided.

  The depth information correction unit 50 receives the depth amount and depth boundary pixel of each pixel of the input image acquired by the depth information acquisition unit 20, and determines the depth amount of the peripheral pixels of the depth boundary pixel including the depth boundary pixel as the depth boundary. The pixel is replaced with the minimum depth amount among the depth amounts of the peripheral pixels including the depth boundary pixel, and is output to the determination unit 30.

[2-2. Operation]
FIG. 10 is an explanatory diagram of depth information correction processing of the depth information correction unit according to the second embodiment.

  FIG. 10A1 illustrates a case where there are a foreground having a depth amount 20 and a background having a depth amount 60 as an input image. FIG. 10 (a2) shows the pixels located in the area surrounded by the broken line in FIG. 10 (a1) and the depth amount each pixel has.

  FIG. 10 (b1) shows a state where the range of the foreground having the depth amount 20 shown in FIG. 10 (a1) is enlarged. FIG. 10B2 shows the pixels located in the area surrounded by the broken line in FIG. 10B1 and the corrected depth amount of each pixel.

  The depth information correction unit 50 receives the depth amount and the depth boundary pixel of each pixel of the input image acquired by the depth information acquisition unit 20, and is the smallest of the nine pixels including the depth boundary pixel 1 pixel and the surrounding 8 pixels. The depth amount is detected, and the depth amount of nine pixels including one depth boundary pixel and the surrounding eight pixels is corrected to the detected minimum depth value, and is output to the determination unit 30.

[2-3. Effect]
This is because the feature amount of the boundary portion changes according to the depth value of the near object. By calculating the boundary pixel as an object on the front side in the determination unit 30, the degree of blur can be accurately obtained.

In the second embodiment, the depth information correction unit 50 corresponds to an example of a depth correction unit.
(Other embodiments)
FIG. 11 is a block diagram illustrating a configuration of an image processing apparatus according to another embodiment.

  The difference from the image processing apparatus 1 of FIG. 1 is that an area specifying unit 60 is further provided.

  The area specifying unit 60 specifies a target area to be subjected to image quality processing of the input image, and outputs the target area to the determining unit 30. The determination unit 30 receives a target area on which the image quality processing of the input image is performed from the area specifying unit 60, determines the degree of blur of each pixel of the input image in the target area, and outputs it to the processing unit 40.

  FIG. 12 is an explanatory diagram of region specifying processing of the region specifying unit according to another embodiment.

  FIG. 12 is an example in the case where there is information other than the captured image, such as captions, in the upper right and lower of the input image.

  In general, when an input image is composed of a caption and a captured image, the caption is superimposed on the captured image by image processing after capturing the captured image, and thus the degree of blur is smaller than that of the captured image. Therefore, the processing unit 40 performs image quality processing with most of the pixels located in the captured image area as pixels with a high degree of blur.

  For this reason, the region specifying unit 60 specifies a target region on which the image quality processing of the input image is to be performed, and outputs the target region to the determining unit 30. The determination unit 30 receives a target area on which the image quality processing of the input image is performed from the area specifying unit 60, determines the degree of blur of each pixel of the input image in the target area, and outputs it to the processing unit 40. The processing unit 40 performs image quality processing that reduces the sense of depth by performing different image quality processing on each pixel of the input image based on the degree of blur of each pixel of the input image determined by the determination unit 30. be able to.

  Note that various known methods can be applied as a method for detecting an area outside the target area for performing image quality processing of an input image such as captions.

  In other embodiments, the region specifying unit 60 corresponds to an example of a region specifying unit.

  FIG. 13 is a block diagram illustrating a configuration of an image processing apparatus according to another embodiment.

  The difference from the image processing apparatus 1 of FIG. 1 is that an image switching detection unit 70 is further provided.

  The image switching detection unit 70 detects switching of the input image and outputs the detection result to the determination unit 30. The switching of input images referred to here means switching from SD video such as DVD to HD video such as BD, scene change, and the like.

  The determining unit 30 obtains a difference between the depth amount of the pixel having the largest feature amount in one frame of the input image and the depth amount of the other pixels of the one frame, and based on the difference, each pixel of the one frame The degree of blur may be determined and output to the processing unit 40.

  Further, the determination unit 30 obtains a difference between the depth amount of the pixel having the largest feature amount and the depth amounts of the other pixels of the plurality of frames in the plurality of frames of the input image, and based on the difference, The degree of blur of each pixel may be determined. In the latter case, the determination unit 30 may reset the feature amount of the pixel having the largest feature amount in a plurality of frames of the input image based on the switching of the input image. In the former case, there is not always a pixel with a small degree of blur in all frames of the input image. If the degree of blur is high in all pixels in one frame, there is a possibility that the degree of blur is determined to be small in all pixels in the one frame. Therefore, the feature amount of the pixel having the largest feature amount in a plurality of frames of the input image may be stored, and the degree of blur of each frame may be determined based on the feature amount.

  In other embodiments, the image switching detection unit 70 corresponds to an example of an image switching detection unit.

  As described above, the embodiments have been described as examples of the technology in the present disclosure. For this purpose, the accompanying drawings and detailed description are provided.

  Accordingly, among the components described in the accompanying drawings and the detailed description, not only the components essential for solving the problem, but also the components not essential for solving the problem in order to illustrate the above implementation. May also be included. Therefore, it should not be immediately recognized that these non-essential components are essential as those non-essential components are described in the accompanying drawings and detailed description.

  Moreover, since the above-mentioned embodiment is for demonstrating the technique in this indication, a various change, replacement, addition, abbreviation, etc. can be performed in a claim or its equivalent range.

  The present disclosure is useful as an image processing technique that can perform image quality processing that reduces the loss of depth.

1, 2, 3, 4 Image processing device 10 Feature amount detection unit 11 First-order differential filter (vertical)
12 First derivative filter (horizontal)
DESCRIPTION OF SYMBOLS 13 Primary differential value calculation part 14 Secondary differential filter 15 Absolute value calculation part 20 Depth information acquisition part 30 Determination part 31 Feature quantity distribution calculation part 32 Feature quantity distribution normalization part 33 Blur degree calculation part 40 Processing part 50 Depth information correction part 60 area specifying unit 70 image switching detecting unit

Claims (9)

A feature amount detector that detects a feature amount of each pixel of the input image;
A depth information acquisition unit for acquiring a depth amount representing a position in the depth direction of each pixel of the input image;
A determining unit that determines a degree of blur of each pixel of the input image based on a difference in depth amount from the depth amount of the pixel having the largest feature amount;
A processing unit that performs different image quality processing for each pixel of the input image based on the degree of blur;
An image processing apparatus comprising: The determination unit
A feature amount distribution generation unit that generates a distribution of the feature amount with respect to the depth amount of each pixel of the input image;
Based on the distribution of the feature amount, the pixel having the largest feature amount has the lowest blur degree, and the pixel having the largest difference in depth amount from the depth amount of the pixel having the largest feature amount determines the blur degree higher. A blur degree determination unit to perform,
The image processing apparatus according to claim 1, further comprising: The depth information acquisition unit
Based on the difference in depth amount between each pixel of the input image, the depth boundary pixel located at the boundary between the foreground and the background of the input image is identified,
The determination unit
Identify pixels within the area bounded by the depth boundary pixels;
The degree of blur of the depth boundary pixels is set higher or lower than the degree of blur of pixels in the area surrounded by the depth boundary pixels;
The image processing apparatus according to claim 1. An area specifying unit for specifying a target area for image quality processing of the input image;
The determining unit determines a degree of blur of each pixel of the input image in the target region;
The image processing apparatus according to claim 1. A depth information correction unit;
The depth information acquisition unit
Detecting a depth boundary pixel located at a boundary between the foreground and the background of the input image based on a difference in depth amount between each pixel of the input image;
The depth information correction unit
Replacing the depth amount of the peripheral pixels of the depth boundary pixel including the depth boundary pixel with the minimum depth amount among the depth amounts of the peripheral pixels of the depth boundary pixel including the depth boundary pixel;
The image processing apparatus according to claim 1. The determination unit
Resetting the depth amount of the pixel having the largest feature amount for each frame of the input image;
The image processing apparatus according to claim 1. The determination unit
Maintaining the depth amount of the pixel having the largest feature amount while updating the plurality of frames of the input image;
The image processing apparatus according to claim 1. An image switching detection unit for detecting a scene change of the input image;
The determination unit
When the scene change of the input image is detected, the depth amount of the pixel having the largest feature amount is reset.
The image processing apparatus according to claim 1. A feature amount detection step for detecting a feature amount of each pixel of the input image;
A depth information acquisition step of acquiring a depth amount representing a position in the depth direction of each pixel of the input image;
A determination step of determining a degree of blur of each pixel of the input image based on a difference in depth amount from the depth amount of the pixel having the largest feature amount;
A processing step of performing different image quality processing for each pixel of the input image based on the degree of blur;
An image processing method comprising: