JP2010073074A - Image signal processor, image display device, video recording/reproducing device, and image signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image signal processor or the like more suitably heightening resolution of an image. <P>SOLUTION: The image signal processor calculates and estimates a blur degree in each pixel or in each partial area of an input image, determines a parameter used for resolution heightening processing about the pixel or the partial area based on the blur degree calculated in estimation processing of the blur degree, and performs the resolution heightening processing to the input image in each pixel or in each partial area based on the parameter determined in parameter determination processing. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for increasing the resolution of an image.

  Recent television receivers have become larger in screen size and do not display image signals input from broadcasting, communication, storage media, etc., as they are, but display them by increasing the number of pixels in the horizontal and vertical directions by digital signal processing. It is generally done. In this case, the resolution cannot be increased only by increasing the number of pixels by a generally known low-pass filter using a sinc function, a spline function, or the like.

  Therefore, Non-Patent Document 1 discloses a super-resolution technique for increasing the resolution by combining a plurality of input image frames (hereinafter abbreviated as frames) into one frame.

Sina Farsiu, M. Dirk Robinson, Michael Elad, Peyman Milanfar “Fast and Robust Multiframe Super Resolution”, IEEE Transactions on Image Processing, VOL. 13, NO. 10, October 2004

  However, the high resolution processing by the technique described in Non-Patent Document 1 acquires the imaging conditions such as the blurring condition and the influence of motion at the time of image capturing, and the high resolution processing is performed using the imaging conditions. I do.

  Then, when only the image signal is acquired without acquiring the imaging condition and the resolution is increased, the imaging condition cannot be used, and the resolution increasing process based on an erroneous condition may be performed. . At this time, there is a problem in that high-resolution processing is applied even to an area that is not focused at the time of imaging, and the resolution is increased and the perspective of the image is lost.

  The present invention has been made in view of the above problems, and an object of the present invention is to increase the resolution of an image while maintaining the perspective of the image for achieving higher resolution of the image.

  The embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, it is possible to improve the resolution of an image more suitably.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, an image signal processing apparatus and an image signal processing method thereof according to Embodiment 1 of the present invention will be described.

FIG. 1 shows an image signal processing apparatus (100) according to Embodiment 1 of the present invention. First, the input image (101) input to the image signal processing device (100) is input to the blur estimation unit (102), and the blur estimation unit (102) performs image blur for each region in the input image. Estimate and calculate the degree of blur.
Next, the parameter determination unit (103) determines a parameter to be used for the high resolution processing based on the blur level estimated by the blur level estimation unit (102). Further, the parameter determined by the parameter determination unit (103) is input to the resolution enhancement processing unit (104), and the resolution enhancement processing unit (104) increases the resolution of the input image using this parameter. Perform high resolution processing. Since the high resolution processing in the high resolution processing unit (104) may use the processing described in Non-Patent Document 1, for example, the description thereof is omitted.

  Here, as a specific example of the blur estimation process in the blur estimation unit (102), there is a method for estimating the blur by calculating the texture amount of the image. The texture amount of the image can be calculated by, for example, a method of calculating the variance of the image from surrounding pixels. In areas where the amount of texture calculated in this way is large, the area where the image is clear, i.e., the area where the degree of blur is small, and on the contrary, in areas where the amount of texture is small, the area where the image is blurred, i.e., where the degree of blur is large. It can be estimated that it is a region. The blur estimation process may be performed for each partial region in the screen or may be performed for each pixel.

  Here, an example of the blur level estimation result of the blur level estimation process of the blur level estimation unit (102) will be described with reference to FIG.

  In the example of FIG. 2, for example, the subject in the area A is a person, and the area A has a focus during imaging. The subject in the region B is a vase behind the person, and the focus of the region B at the time of imaging is shifted from the region A. Furthermore, it is assumed that the subject in the area C is a background that is further behind the person and the vase, and the focus at the time of imaging is shifted from the areas A and B in the area C.

  When the image of FIG. 2 as described above is used as the input image, according to the blur level estimation process in the blur level estimation unit (102), the region A is estimated to have a low blur level (clear image), and the region B Is estimated to be medium blur (an image that is more blurred than region A, an image that is less clear than region A), and region C has a greater blur level (an image that is more blurred than region B and an image that is less clear than region B). Can be estimated.

  In the above example, in the blur estimation process of the blur estimation unit (102), the texture amount is used to estimate the blur amount of the image. For example, the edge information is calculated and the calculated edge information is used. It may be estimated by other methods such as determining whether an image is clear or blurred.

  Next, as a specific example of the parameter determination process of the parameter determination unit (103), there is a method using a blur recovery transfer function as a parameter.

  That is, when the processing described in Non-Patent Document 1 is used for the resolution enhancement processing in the resolution enhancement processing unit (104), a transfer function for recovering image blurring that occurs during imaging is required as a parameter. . Generally, this transfer function is a low-pass filter coefficient. When the low-pass filter coefficient is set to be strong in the low-pass filter process, the blur recovery effect of the high-resolution process is strong. Conversely, when the low-pass filter process is set to be weak, the blur recovery effect of the high-resolution process is weak.

  The parameter determination unit (103) according to the first embodiment changes the low-pass filter coefficient for each partial region in the image according to the blur level calculated by the blur level estimation unit (102). This makes it possible to change the strength of the low-pass filter in the high resolution processing of the high resolution processing unit (104) for each partial region in the image, and this occurs during imaging while maintaining the perspective of the image. Image blur can be recovered and the image can be increased in resolution.

  When the blur level estimation process of the blur level estimation unit (102) is a blur level estimation process by calculating the texture amount, the parameter determination unit (103) calculates the texture amount calculated by the blur level estimation unit (102). A region with a large amount of noise sets a low-pass filter strongly, and a region with a small amount of texture sets a low-pass filter weakly.

  In the above description, the blur recovery transfer function is given as an example of the parameter used for the resolution enhancement processing in the resolution enhancement processing unit (104) .However, the parameter for controlling the image blur recovery effect in the other resolution enhancement processing is described. It may be used.

  Note that the blur level estimation processing may be performed for each partial region in the screen or may be performed for each pixel.

  Finally, the high resolution processing unit (104) performs high resolution processing of the input image using the parameters calculated by the parameter determination unit (103). The resolution enhancement process may be performed for each partial region in the screen or may be performed for each pixel.

  When the parameter determined by the parameter determination unit (103) is a low-pass filter coefficient, the resolution enhancement processing unit (104) uses the low-pass filter coefficient determined by the parameter determination unit (103). High resolution processing including processing is performed.

  Next, FIG. 5 shows a processing flow of image processing by the image signal processing apparatus (100) described above. In the image signal processing device (100), the blur level estimation unit (102) calculates and estimates the blur level of the image for each partial region or one pixel in the screen of the input image (step 501). Next, the parameter determination unit (103) performs the resolution enhancement processing unit (104) for each partial region or one pixel of the input image based on the blur level calculated and estimated by the blur level estimation unit (102). Parameters used for the high resolution processing are determined (step 502). Finally, the resolution enhancement processing unit (104) performs a process for increasing the resolution of the input image using the parameters determined by the parameter determination unit (103) (step 503).

  According to the image signal processing apparatus according to the first embodiment described above, a partial area or pixel that should be improved in blur and a partial area or pixel that should be left out of blur are determined from the degree of blur of the input image. Therefore, it is possible to perform high resolution processing corresponding to the degree of blurring on the input image.

  That is, according to the image signal processing apparatus according to the first embodiment described above, the blurring degree is estimated for each partial region or one pixel of the input image, and the high resolution processing is changed based on the estimated blurring degree. By doing so, it is possible to perform high resolution processing while maintaining the perspective of the image. Therefore, it is possible to improve the resolution of the image more suitably.

FIG. 3 shows an image display apparatus according to Embodiment 2 of the present invention. The image display apparatus according to the present embodiment is an image display apparatus configured to include an image signal processing unit that performs the image signal processing described in the first embodiment.
In the figure, an image display device (300) includes, for example, an input unit (301) for inputting a broadcast signal, image content, image content, etc. via a broadcast wave including a television signal, a network, etc. 301), a recording / playback unit (302) for recording or playing back the content input from the content, a content storage unit (303) for recording the content by the recording / playback unit (302), and an image signal played back by the recording / playback unit (302). The image signal processing unit (304) for performing the image signal processing described in the first embodiment, the display unit (305) for displaying the image signal processed by the image signal processing unit (304), and recording / playback An audio output unit (306) for outputting an audio signal reproduced by the unit (302), a control unit (307) for controlling each component of the image display device (300), and a user operating the image display device (300) A user interface unit (308) for performing
The detailed configuration and operation of the image signal processing unit (304) are the same as those of the image signal processing device (100) described in the first embodiment, and thus description thereof is omitted.
Since the image display device (300) includes the image signal processing unit (304) that is the image signal processing device described in the first embodiment, the image signal input to the input unit (301) The resolution can be increased and displayed on the display unit (305) while being held.

  Further, when reproducing the image content stored in the content storage unit (303), it is possible to increase the resolution while displaying the perspective of the image and display it on the display unit (305).

  Further, the image signal processing unit (304) may be included in the recording / playback unit (302) to perform the above-described image signal processing during recording. In this case, since it is not necessary to perform the above-described image signal processing during reproduction, the processing load during reproduction can be reduced.

  According to the image display device according to the second embodiment of the present invention described above, the image display device having the effects shown in the first embodiment can be realized. Therefore, the image can be displayed with a higher resolution while maintaining the perspective of the image.

FIG. 4 shows a recording / playback apparatus according to Embodiment 3 of the present invention. The recording / playback apparatus according to the present embodiment is a recording / playback apparatus configured to include an image signal processing unit that performs the image signal processing of the first embodiment.
In the figure, a recording / playback apparatus (400) includes, for example, an input unit (401) for inputting a broadcast signal, image content, image content, etc. via a broadcast wave including a television signal, a network, etc. A recording / playback unit (402) that records or plays back the content input from the unit (401), a content storage unit (403) that records the content by the recording / playback unit (402), and a recording / playback unit (402) that plays back An image signal processing unit (404) that performs image signal processing described in the first embodiment on the image signal, and an image output that outputs the image signal processed by the image signal processing unit (404) to another device or the like Unit (405), an audio output unit (406) for outputting the audio signal reproduced by the recording / reproducing unit (402) to other devices, and a control unit (407) for controlling each component of the recording / reproducing device (400) ) And a user interface for the user to operate the recording / playback device (400) (408) comprises a like.
The recording / playback apparatus (400) includes the image signal processing unit (404) which is the image signal processing apparatus described in the first embodiment, so that the image signal with higher resolution while maintaining the perspective of the image can be used as another image signal. It can be output to a device or the like.

Further, the image signal processing unit (404) may be included in the recording / playback unit (402) to perform the above-described image signal processing during recording. In this case, since it is not necessary to perform the above-described image signal processing during reproduction, the processing load during reproduction can be reduced.
According to the recording / reproducing apparatus according to the third embodiment of the present invention described above, the recording / reproducing apparatus having the effects shown in the first embodiment can be realized. Therefore, it is possible to record, reproduce or output the image with a high resolution while maintaining the perspective of the image.

It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 3 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention.

Explanation of symbols

101 Input image
102 Blur level estimation unit
103 Parameter determination section
104 High resolution processing section
105 Output image
201 Image texture amount calculator
202 Transfer function determination unit for blur recovery
300 Image display device
301 Input section
302 Recording / playback unit
303 Content storage
304 Image signal processor
305 Display
306 Audio output section
307 Control unit
308 User interface
400 Recording / playback device
401 input section
402 Recording / playback unit
403 content storage
404 Image signal processor
405 Image image output unit
406 Audio output unit
407 Control unit
408 User interface

Claims (10)

A blur estimation unit that calculates blur for each partial region or pixel of the input image;
A parameter determining unit that determines a parameter to be used for high resolution processing for the partial region or the pixel, based on the blur level calculated by the blur level estimation unit;
An image signal processing apparatus comprising: a resolution enhancement processing unit that performs resolution enhancement processing on the input image for each partial region or for each pixel based on the parameter determined by the parameter determination unit.   The image signal processing apparatus according to claim 1, wherein the blur level estimation unit calculates a texture amount of the image as the blur level for each partial region or pixel of the input image.   The image signal processing apparatus according to claim 1, wherein the parameter determined by the parameter determination unit is a low-pass filter coefficient of low-pass filter processing included in the resolution enhancement processing performed in the resolution enhancement processing unit. An image signal processing unit which is the image signal processing device according to claim 1;
An image display apparatus comprising: a display unit configured to display an image based on an image signal subjected to image processing in the image signal processing unit. An accumulator for accumulating input images;
An image signal processing unit as an image signal processing device according to claim 1, comprising: an image signal processing unit that performs image processing on an image reproduced from the storage unit. . A blur estimation step for calculating blur for each partial area or pixel of the input image;
A parameter determining step for determining a parameter to be used for high resolution processing for the partial region or the pixel based on the blurring degree calculated in the blurring degree estimating step;
An image signal processing method comprising: a resolution enhancement processing step of performing resolution enhancement processing on the input image for each partial region or for each pixel based on the parameter determined in the parameter determination step.   The image signal processing method according to claim 6, wherein in the blur level estimation step, a texture amount of an image is calculated as the blur level for each partial region or pixel of the input image.   The image signal processing method according to claim 6, wherein the parameter determined in the parameter determination step is a low-pass filter coefficient of a low-pass filter process included in the resolution enhancement process performed in the resolution enhancement process step. The image signal processing method according to claim 6,
And a display step for displaying an image on the basis of the image signal subjected to the high resolution processing in the high resolution processing step. An accumulation step for accumulating input images;
A reproduction step of reproducing the image accumulated in the accumulation step;
A blur estimation step for calculating blur for each partial region or pixel of the image reproduced in the reproduction step;
A parameter determining step for determining a parameter to be used for high resolution processing for the partial region or the pixel based on the blurring degree calculated in the blurring degree estimating step;
An image processing method comprising: a resolution enhancement processing step for performing a resolution enhancement processing on the image reproduced by the reproduction step for each partial region or for each pixel based on the parameter determined in the parameter determination step. Signal processing method.

Images