JP2013141198A - Image processing device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of sufficiently improving moving image visibility even if a pixel value after a high-frequency component has been emphasized is out of a prescribed range.SOLUTION: An image processing device comprises: frame rate conversion means; emphasis means for generating a high-pass emphasis image signal by emphasizing a sub-frame image signal; suppression means for generating a high-pass suppression image signal by suppressing the sub-frame image signal; adjustment means for compensating the high-pass suppression image signal by limiting the high-pass emphasis image signal; and output means for alternately outputting the limited high-pass emphasis signal and the compensated high-pass suppression image signal to a display device per sub-frame. The adjustment means generates a compensation value so that a compensation value added to a signal in a moving area among high-pass suppression image signals becomes a value smaller than a variation of a pixel value by limit processing.

Description

  The present invention relates to an image processing apparatus and a control method thereof.

  There is a technique for improving the visibility of moving images by suppressing flicker by increasing the frame rate of an input image signal twice and alternately outputting a high frequency emphasis subframe and a high frequency suppression subframe. The high frequency emphasis subframe is a subframe in which spatial high frequency components are emphasized, and the high frequency suppression subframe is a subframe in which spatial high frequency components are suppressed. Conventionally, when the pixel value of the high-frequency emphasis subframe becomes a value outside the specified range, a processing method for limiting the pixel value to a value within the specified range has been proposed (for example, Patent Documents 1 and 2). reference). Also, a processing method has been proposed in which the amount of change in pixel value due to the above limitation is added to the pixel value of the high-frequency suppression subframe (see, for example, Patent Document 3). According to the latter method, the average image signal of the image signal of the high frequency enhancement subframe (high frequency enhancement image signal) and the image signal of the high frequency suppression subframe (high frequency suppression image signal) is made equal to the input image signal. be able to. Therefore, a still image that is equal to the image based on the input image signal and has no distortion can be displayed.

JP 2006-184896 A JP 2006-195405 A US Patent Application Publication No. 2006/0227249

However, in the techniques disclosed in Patent Documents 1 and 2, the pixel value of the high-frequency emphasis frame is limited, and the amount of change in the pixel value due to the limitation is discarded. Therefore, there is a problem that the average image signal of the high-frequency emphasized image signal and the high-frequency suppressed image signal is not equal to the input image signal, and the display image is distorted.
In the technique disclosed in Patent Document 3, since the average image signal of the high-frequency emphasized image signal and the high-frequency suppressed image signal can be made equal to the input image signal, a still image without distortion can be displayed. However, the amount of change in the pixel value due to the restriction of the pixel value of the high frequency emphasis subframe is added to the pixel value of the high frequency suppression frame, that is, the high frequency component is added to the high frequency suppression image signal. The high band suppression effect of the band suppression frame is reduced. As a result, the effect of improving the moving image visibility is reduced.

  An object of the present invention is to provide a technique capable of sufficiently improving the moving image visibility even when the pixel value after the high-frequency component is emphasized becomes a value outside the specified range.

The image processing apparatus of the present invention
Frame rate conversion means for generating a plurality of subframes from a frame of the input image signal and increasing the frame rate of the input image signal;
An emphasis unit that generates a high-frequency emphasized image signal by performing an emphasis process that emphasizes a spatial high-frequency component on the image signal of the subframe;
Suppression means for generating a high-frequency-suppressed image signal by applying a suppression process for suppressing a spatial high-frequency component to the image signal of the subframe;
The high-frequency emphasized image signal is subjected to a restriction process that restricts a pixel value to a value within a specified range, and a compensation value that compensates for the amount of change in the pixel value due to the restriction process is added to the high-frequency suppression image signal. Adjusting means for performing compensation processing;
An output means for alternately outputting the high-frequency emphasized image signal subjected to the restriction process and the high-frequency suppressed image signal subjected to the compensation process to the display device every subframe;
Have
The adjusting unit generates the compensation value so that a compensation value to be added to a signal in a moving region of the high-frequency-suppressed image signal is smaller than a change amount of a pixel value due to the restriction process. Features.

The control method of the image processing apparatus of the present invention includes:
A frame rate conversion step of generating a plurality of subframes from a frame of the input image signal and increasing a frame rate of the input image signal;
An emphasis step for generating a high-frequency emphasized image signal by applying an emphasis process that emphasizes a spatial high-frequency component to the image signal of the subframe;
A suppression step of generating a high-frequency-suppressed image signal by applying a suppression process that suppresses a spatial high-frequency component to the image signal of the subframe;
The high-frequency emphasized image signal is subjected to a restriction process that restricts the pixel value to a value within a specified range, and a compensation value that compensates for the amount of change in the pixel value due to the restriction process is added to the high-frequency-suppressed image signal. An adjustment step for performing compensation processing;
An output step of alternately outputting the high-frequency emphasized image signal subjected to the restriction processing and the high-frequency suppression image signal subjected to the compensation processing to the display device every subframe;
Have
In the adjustment step, generating a compensation value so that a compensation value to be added to a signal in a moving region of the high-frequency-suppressed image signal is smaller than a change amount of a pixel value due to the restriction process. Features.

  According to the present invention, it is possible to sufficiently improve the moving image visibility even when the pixel value after the high-frequency component is emphasized becomes a value outside the specified range.

1 is a diagram illustrating an example of a functional configuration diagram of an image processing apparatus according to Embodiment 1. FIG. The figure which shows an example of the function block diagram of the time direction filter part which concerns on Example 1. FIG. FIG. 6 is a diagram for explaining an example of a configuration of a high frequency emphasis unit according to the first embodiment. FIG. 6 is a diagram illustrating image processing according to the first embodiment. FIG. 10 is a diagram illustrating an example of a functional configuration diagram of an image processing apparatus according to a second embodiment. The figure which shows an example of the function block diagram of the compensation suppression control part which concerns on Example 2. FIG. FIG. 6 is a diagram illustrating image processing according to the second embodiment. FIG. 10 is a diagram illustrating an example of a functional configuration diagram of an image processing apparatus according to a third embodiment. The figure which shows an example of the function block diagram of the compensation suppression control part which concerns on Example 3. FIG. The figure which shows an example of the function block diagram of the stationary edge detection part which concerns on Example 3. FIG. The figure which shows an example of the function block diagram of the tap control LPF which concerns on Example 3. FIG. The figure explaining the operation | movement of the suppression area | region determination which concerns on Example 3. FIG. FIG. 6 is a diagram illustrating image processing according to the third embodiment.

<Example 1>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 1 of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating an example of a functional configuration of the image processing apparatus according to the first embodiment.
The image processing apparatus according to the first embodiment includes a frame converter unit 101, a time direction filter unit 102, a first selection unit 103, a high frequency suppression unit 104, a high frequency enhancement unit 105, a limit unit 106, an addition unit 107, and a second selection. Part 108 and the like.

  The frame converter 101 generates a plurality of subframes from a frame of an image signal (input image signal) input from the outside, and increases the frame rate of the input image signal (frame rate conversion). Specifically, the frame converter unit 101 divides one frame period of the input image signal into periods of two subframes (a first subframe and a second subframe next to the first subframe). Double the frame rate of the input image signal. Then, for each frame of the input image signal, the image signal of that frame is output as a sub-frame image signal in the period of two sub-frames obtained by dividing the period of the frame. The image signal of each subframe is input to the time direction filter unit 102 and the first selection unit 103. In addition, the frame converter unit 101 outputs a subframe determination signal indicating whether the output image signal is an image signal of the first subframe or an image signal of the second subframe. The subframe determination signal is input to the first selection unit 103 and the second selection unit 108.

The time direction filter unit 102 performs time direction filter processing (smoothing processing) that reduces the amount of change in pixel value between subframes on the input subframe image signal, thereby obtaining the time direction smoothed image signal. Generate. Then, the time direction filter unit 102 outputs the generated time direction smoothed image signal to the first selection unit.
FIG. 2 shows an example of a functional configuration of the time direction filter unit 102. The time direction filter unit 102 includes a frame delay unit 109, an average image generation unit 110, and the like. The frame delay unit 109 delays the input image signal by one subframe period and outputs it to the average image generation unit 110. The average image generation unit 110 calculates, for each pixel, an average value of the two image signals by calculating an average value of the image signal input from the frame converter unit 101 and the image signal input from the frame delay unit 109. And output as a time direction smoothed image signal.
In this embodiment, the time direction smoothed image is an average image signal of image signals of two subframes separated by one subframe period, but the time direction smoothed image is not limited to this. For example, the time direction smoothed image signal may be an average image signal of image signals of four subframes, or may be an image signal generated by weighted synthesis of image signals of a plurality of subframes. The time direction filter unit 102 may be configured with an IIR filter.

  The first selection unit 103 selects and outputs the time direction smoothed image signal as the image signal of the first subframe based on the subframe discrimination signal output from the frame converter unit 101. The first selection unit 103 selects and outputs the image signal output from the frame converter unit 101 as the image signal of the second subframe.

  The high-frequency suppressing unit 104 performs filtering processing (suppression processing) that suppresses spatial high-frequency components on the sub-frame image signal (the image signal selected by the first selecting unit 103), thereby generating a high-frequency suppressed image signal. Is output to the high frequency emphasis unit 105 and the addition unit 107 (second generation). Note that the high-frequency suppressed image signal can be rephrased as a low-frequency image signal in which the spatial low-frequency component is larger than the high-frequency component. Further, instead of the high frequency suppression unit 104, a low pass filter processing unit that allows low frequency components to pass may be used.

The high-frequency emphasizing unit 105 generates a high-frequency emphasized image signal by performing enhancement processing that emphasizes a spatial high-frequency component on the image signal of the subframe (the image signal selected by the first selection unit 103), Output to the limit unit 106 (first generation). Note that the high-frequency emphasized image signal can be rephrased as a high-frequency image signal having more spatial high-frequency components than low-frequency components. Further, instead of the high frequency emphasizing unit 105, a high pass filter processing unit that allows high frequency components to pass therethrough may be used.
FIG. 3 is a diagram for explaining an example of the configuration of the high frequency emphasizing unit 105. The high frequency emphasizing unit 105 doubles the pixel value of the input image signal. Then, the high frequency emphasizing unit 105 generates a high frequency emphasized image signal by subtracting the high frequency suppressed image signal output from the high frequency suppressing unit 104 from the image signal whose pixel value is doubled.
Note that the configuration of the high frequency emphasis unit 105 is not limited to the configuration shown in FIG. For example, the high frequency emphasizing unit 105 may generate a high frequency emphasized image signal by performing filter processing using a high frequency emphasizing filter on the image signal.

As can be seen from the configuration of the high frequency emphasis unit 105, the pixel value of the high frequency emphasized image signal may be a value outside the specified range. Specifically, the pixel value of the high-frequency emphasized image signal may be a value that is larger than the maximum value in the specified range or a value that is smaller than the minimum value in the specified range.
The limit unit 106 subjects the high-frequency emphasized image signal output from the high-frequency emphasizing unit 105 to a restriction process that restricts the pixel value to a value within a specified range, and the high-frequency emphasized image signal subjected to the restriction process Is output to the second selection unit.
Further, the limit unit 106 generates a compensation value that compensates for the amount of change in the pixel value due to the limiting process, and outputs the compensation value to the addition unit 107. In the present embodiment, the amount of change in the pixel value due to the restriction process (the pixel value that is discarded) is output to the adder 107 as a compensation value. For example, when the image signal is an 8-bit signal and the specified range of pixel values (gradation values) is in the range of 0 to 255, the high-frequency enhanced image signal output from the high-frequency enhancing unit 105 Are limited to values in the range of 0-255. When the pixel value of the high-frequency emphasized image signal output from the high-frequency emphasizing unit 105 is 300, the pixel value is limited to 255 and the limit value is 45.

The adding unit 107 performs compensation processing for adding a compensation value to the high frequency suppressed image signal output from the high frequency suppressing unit 104, and the high frequency suppressed image signal subjected to the compensation processing is supplied to the second selecting unit 108. Output. In the present embodiment, the limit value output from the limit unit 106 is added to the high frequency suppressed image signal output from the high frequency suppressing unit 104 and is output to the second selection unit 108.
As described above, in this embodiment, the limit unit 105 and the addition unit 107 adjust the pixel value of the high-frequency emphasized image signal and the pixel value of the high-frequency suppressed image signal.

  The second selection unit 108 includes a high-frequency emphasized image signal subjected to the restriction process (an image signal output from the limit unit 106) and a high-frequency suppression image signal subjected to the compensation process (output from the addition unit 107). Image signal) is alternately output to the display device every subframe. Specifically, the second selection unit 108 selects the image signal output from the addition unit 107 in the first subframe based on the subframe determination signal output from the frame converter unit 101, and outputs it to the display device. Output. In the second subframe, the second selection unit 108 selects the image signal output from the limit unit 106 and outputs it to the display device.

  As described above, the first selection unit 103 selects and outputs the time direction smoothed image signal in the first subframe and the image signal output from the frame converter unit 101 in the second subframe. For this reason, in this embodiment, the high-frequency emphasized image signal to be output to the display device is generated by sequentially performing the enhancement process and the restriction process on the image signal output from the frame converter unit 101. Further, the suppression processing and the compensation processing are sequentially performed on the time-direction smoothed image signal, thereby generating a high-frequency suppression image signal to be output to the display device. At this time, the compensation value used in the compensation process is generated using the time-direction smoothed image signal output from the first selection unit 103. Specifically, when the enhancement process and the restriction process are sequentially performed on the time-direction smoothed image signal, the change amount of the pixel value by the restriction process is set as the compensation value.

FIG. 4 shows an example of a comparison between conventional image processing and image processing according to the first embodiment for an image region that moves between frames. In FIG. 4, the horizontal axis represents the pixel position, and the vertical axis represents the pixel value. FIG. 4 shows the state of image processing for a signal in an image area that moves to the right (left as viewed from the viewer) from the previous frame to the current frame.

  In conventional image processing, an input image signal is subjected to suppression processing and compensation processing in order, thereby generating a high-frequency suppressed image signal to be output to the display device. At this time, in the compensation process, the amount of change in the pixel value due to the restriction process performed when generating the high-frequency emphasized image signal to be output to the display device is used as the compensation value. That is, in the conventional image processing, by adding the amount of change in the pixel value due to the restriction processing performed when generating the high-frequency emphasized image signal output to the display device to the input image signal subjected to the suppression processing. Then, a high-frequency suppression image signal to be output to the display device is generated. Therefore, the high frequency component of the high frequency suppressed image signal output to the display device may not be sufficiently suppressed. As a result, a double image is visually recognized when a moving region is viewed.

In Example 1, the compensation value used in the suppression process performed when generating the high-frequency suppression image signal to be output to the display device is generated using the time-direction smoothed image signal. Specifically, when the enhancement process and the restriction process are sequentially performed on the time-direction smoothed image signal, the change amount of the pixel value by the restriction process is set as the compensation value. The time-direction smoothed image signal is a signal in which the amount of change in pixel value from the previous subframe in a region with motion is reduced. For this reason, the compensation value generated from the time-direction smoothed image signal (the compensation value added to the signal in the region with motion) is based on the restriction processing performed when the high-frequency emphasized image signal to be output to the display device is generated. The value is smaller than the change amount of the pixel value. Then, compensation for the amount of change is suppressed in a region with motion, and a high-frequency suppressed image signal in which a spatial high-frequency component in the region with motion is suppressed is obtained. As a result, it is suppressed that the double image is visually recognized in the region where the pixel value has changed from the previous sub-frame, that is, the region with motion, and the moving image visibility is improved. In the example of the image signal input shown in FIG. 4, since the total amount of the positive compensation value is zero and the total amount of the negative compensation value does not change, the compensation in the region with motion is suppressed (suppressed) to about half. The Although depending on the pattern of the image, when the compensation value is generated from the time direction smoothed image signal of the present embodiment, the compensation in the region with motion is suppressed to about half or less.
In a region where there is no movement, the time direction smoothed image signal is equal to the image signal output from the frame converter unit 101. Therefore, in a region where there is no movement, as a high-frequency suppression image signal output to the display device, a high-frequency suppression image signal (high-frequency suppression image signal output to the display device) obtained by the processing described in the conventional example in FIG. The same signal is obtained. In addition, the high-frequency emphasized image signal of the non-motion region output to the display device is an image obtained by performing enhancement processing and limit processing on the image signal output from the frame converter unit 101, as in the case of the motion region. Signal. That is, in a region where there is no motion, the same signal as the signal obtained by the processing described in the conventional example in FIG. 4 is obtained as the high-frequency suppression image signal and high-frequency emphasized image signal output to the display device. For this reason, in an area where there is no motion, the average image signal of the high-frequency emphasized image signal and the high-frequency suppressed image signal is made equal to the input image signal as in the conventional case. As a result, a still image that is equal to the image based on the input image signal and has no distortion can be displayed.

As described above, according to the present embodiment, the compensation value to be added to the signal of the moving region in the high frequency suppressed image signal is performed when the high frequency emphasized image signal to be output to the display device is generated. The compensation value is generated so that the value is smaller than the change amount of the pixel value by the limiting process. Specifically, a compensation value is generated using the time direction smoothed image signal. Thereby, even when the pixel value after the high frequency component is emphasized becomes a value outside the specified range, the moving image visibility can be sufficiently improved.
In this embodiment, the high-frequency suppressed image signal to be output to the display device is generated by sequentially performing the suppression process and the compensation process on the time-direction smoothed image signal. However, the present invention is not limited to this. For example, a high frequency suppressed image signal to be output to the display device may be generated by sequentially performing a suppression process and a compensation process on the input image signal. If the compensation value is generated using the time-direction smoothed image signal, the compensation of the pixel value by the compensation process is suppressed in a region where there is a motion, so that the moving image visibility can be sufficiently improved.

<Example 2>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 2 of the present invention will be described with reference to the drawings. In the first embodiment, the method of improving the moving image visibility by suppressing the compensation of the change amount in the region with the motion by generating the correction value using the time direction smoothed image has been described. In the present embodiment, a method for suppressing the compensation of the amount of change in a region with motion based on the motion information of the image signal will be described.

FIG. 5 is a diagram illustrating an example of a functional configuration diagram of the image processing apparatus according to the second embodiment. The image processing apparatus includes a frame converter unit 101, a high frequency suppressing unit 104, a high frequency emphasizing unit 105, a limit unit 106, a compensation suppression control unit 211, an adding unit 107, a selecting unit 208, and the like.
The functions of the frame converter unit 101, the high frequency suppressing unit 104, the high frequency emphasizing unit 105, the limit unit 106, and the adding unit 107 are the same as those in the first embodiment.
However, in the present embodiment, the amount of change in the pixel value when the restriction process is applied to the input image signal that has been subjected to the enhancement process is always the compensation value. Specifically, an input image signal is always input to the high frequency emphasis unit 105 as a subframe image signal. Then, the limit unit 106 performs a limit process on the input image signal subjected to the enhancement process, and a change amount of the pixel value by the limit process is set as a compensation value.

The compensation suppression control unit 211 corrects the compensation value output from the limit unit 106 based on the motion information of the image signal, and outputs the correction value to the addition unit 107.
FIG. 6 illustrates an example of a functional configuration of the compensation suppression control unit 211 according to the second embodiment. The compensation suppression control unit 211 includes a frame delay unit 212, a frame difference calculation unit 213, an LPF unit 214, a suppression region determination unit 215, a multiplication unit 216, and the like.

The frame delay unit 212, the frame difference calculation unit 213, the LPF unit 214, and the suppression region determination unit 215 detect a region in which the image signal of the subframe moves.
The frame delay unit 212 outputs the input subframe image signal with a delay of one subframe period.
The frame difference calculation unit 213 calculates the absolute value of the difference between the pixel value of the image signal input from the frame converter unit 101 and the pixel value of the image signal input from the frame delay unit 212 for each pixel. Then, the frame difference calculation unit 213 outputs the absolute value of the difference for each pixel to the LPF unit 214 as a frame difference signal.
The LPF unit 214 performs suppression processing on the frame difference signal generated by the frame difference calculation unit 213, and outputs the frame difference signal subjected to the suppression processing to the suppression region determination unit 215 as a high-frequency suppression frame difference signal.
The suppression region determination unit 215 compares the value of each pixel of the high frequency suppression frame difference signal generated by the LPF unit 214 with a determination threshold (predetermined threshold). And the suppression area | region determination part 215 determines the area | region where a value is more than a determination threshold among high frequency suppression frame difference signals as a movement area (suppression area), and areas other than a suppression area (value is less than a determination threshold) Area) is determined as a non-motion area (non-suppression area). Then, the suppression region determination unit 215 outputs 0 to the multiplication unit 216 as the suppression control signal for the pixel determined to be the suppression region and 1 as the suppression control signal for the pixel determined to be the non-suppression region. As the determination threshold value, for example, a value of about 5% of the pixel value range (specified range) is used. Note that the determination threshold is not limited to this value. The determination threshold value may be changeable and may be appropriately set by the user according to the purpose.

Multiplier 216 reduces the compensation value to be added to the suppression region signal detected by suppression region determination unit 215 among the compensation values generated by limit unit 106. Specifically, the multiplication unit 216
Multiplies the compensation value generated by the limit unit 106 by the suppression control signal generated by the suppression region determination unit 215 and outputs the result to the addition unit 107. Since the value of the suppression control signal is 0 in a region with motion (suppression region), the compensation value generated by the limit unit 106 is set to 0 and output to the addition unit 107. Since the value of the suppression control signal is 1 in a region where there is no motion (non-suppression region), the compensation value generated by the limit unit 106 is output to the addition unit 107 as it is (without being reduced).
In the present embodiment, the suppression control signal is a binary signal of 0 and 1. However, the suppression control signal may be a signal of three or more values by using a plurality of determination thresholds. In this case, for example, by setting the suppression control signal as a signal having three or more values of 0 to 1, the compensation value output from the multiplying unit 216 is set to a value of three or more of the compensation values generated by the 0 to limit unit 106. Can be controlled between.
In the present embodiment, among the compensation values generated by the limit unit 106, the compensation value added to the suppression region is set to 0. However, the present invention is not limited to this. Of the compensation values generated by the limit unit 106, the compensation value to be added to the suppression region may be reduced. The amount of reduction of the compensation value may be increased from the boundary between the non-suppression region and the suppression region toward the inside of the suppression region.

  Based on the subframe discrimination signal output from the frame converter unit 101, the selection unit 208 selects the image signal output from the limit unit 106 in the first subframe, and outputs the image signal to the display device. In addition, in the second subframe, the selection unit 208 selects the image signal output from the addition unit 107 and outputs it to the display device.

FIG. 7 shows an example of comparison between conventional image processing and image processing according to the second embodiment for an image region that moves between frames. In FIG. 7, the horizontal axis represents the pixel position, and the vertical axis represents the pixel value. FIG. 7 shows a state of image processing for a signal in an image area that moves to the right (left as viewed from the viewer) from the previous frame to the current frame, as in FIG.
As described in the first embodiment, in the conventional image processing, the high frequency component of the high frequency suppressed image signal output to the display device may not be sufficiently suppressed. For this reason, a double image is visually recognized when a moving region is viewed.

In the second embodiment, for each pixel, an absolute value of a difference between two consecutive subframes is calculated, and a moving region (suppression region) is detected based on the calculation result. Then, among the generated compensation values, the compensation value to be added to the suppression region is reduced (set to 0). For this reason, compensation for the amount of change is suppressed in a region with motion, and a high-frequency-suppressed image signal in which a spatial high-frequency component in the region with motion is suppressed is obtained. As a result, it is suppressed that the double image is visually recognized in the region where the pixel value has changed from the previous sub-frame, that is, the region with motion, and the moving image visibility is improved.
In addition, since the area | region without a motion is not detected as a suppression area | region, compensation is not suppressed with respect to the area | region without a motion, but the process similar to a prior art example is performed. For this reason, in a static region where there is no movement, the average image signal of the high frequency emphasized image signal and the high frequency suppressed image signal is made equal to the input image signal as in the conventional case. As a result, a still image that is equal to the image based on the input image signal and has no distortion can be displayed.

  As described above, according to the present embodiment, the amount of change in the pixel value when the input image signal subjected to the enhancement process is subjected to the restriction process (that is, the high-frequency emphasized image signal to be output to the display device is generated). The amount of change in the pixel value due to the restriction process performed at the time of the determination is a compensation value. Of the compensation values, the compensation value to be added to the signal in the region with motion is reduced. Thereby, even when the pixel value after the high frequency component is emphasized becomes a value outside the specified range, the moving image visibility can be sufficiently improved.

<Example 3>
Hereinafter, an image processing apparatus and a control method thereof according to Embodiment 3 of the present invention will be described with reference to the drawings.
FIG. 8 is a diagram illustrating an example of a functional configuration of the image processing apparatus according to the third embodiment.
The image processing apparatus according to the third embodiment includes a frame converter unit 101, a time direction filter unit 102, a first selection unit 103, a high frequency suppression unit 104, a high frequency enhancement unit 105, a limit unit 106, a compensation suppression control unit 311 and an addition. Unit 107, second selection unit 108, and the like.
The functions of the frame converter unit 101, the time direction filter unit 102, the first selection unit 103, the high frequency suppression unit 104, the high frequency enhancement unit 105, the limit unit 106, the addition unit 107, and the second selection unit 108 are the same as those in the first embodiment. It is.

The compensation suppression control unit 311 corrects the compensation value output from the limit unit 106 based on the motion information of the image signal, and outputs the correction value to the addition unit 107.
FIG. 9 illustrates an example of a functional configuration of the compensation suppression control unit 311 according to the third embodiment. The compensation suppression control unit 311 includes a frame delay unit 212, a frame difference calculation unit 213, a stationary edge detection unit 317, a tap control LPF unit 314, a suppression region determination unit 215, a multiplication unit 216, and the like. The functions of the frame delay unit 212, the frame difference calculation unit 213, the suppression region determination unit 215, and the multiplication unit 216 are the same as those in the second embodiment.

The frame delay unit 212, the frame difference calculation unit 213, the stationary edge detection unit 317, the tap control LPF unit 314, and the suppression region determination unit 215 detect a region where the image signal of the subframe moves.
The stationary edge detection unit 317 uses the frame difference signal generated by the frame difference calculation unit 213, the image signal input from the frame converter unit 101, and the image signal input from the frame delay unit 212, so that an edge with no motion ( Detect still edges). Then, the stationary edge detection unit 317 outputs the detection result to the tap control LPF unit 314 as a stationary edge detection signal.
FIG. 10 illustrates an example of a functional configuration of the stationary edge detection unit 317 according to the third embodiment. The stationary edge detector 317 includes a stationary pixel detector 318, a first edge detector 319, a second edge detector 320, a common edge detector 321 and a stationary pixel adjacent edge detector 322.
The still pixel detection unit 318 compares the frame difference signal with a still pixel detection threshold (predetermined threshold) for each pixel. And the still pixel detection part 318 detects the pixel whose value is less than a still pixel detection threshold value as a still pixel (pixel without a motion) among frame difference signals. As the still pixel detection threshold value, for example, a value of about 1% of the pixel value range (specified range) is used. Note that the still pixel determination threshold is not limited to this value. The still pixel determination threshold value may be changeable and may be set by the user according to the purpose.
The first edge detection unit 319 extracts a high frequency component (edge component) from the image signal input from the frame converter unit 101 and outputs the high frequency component (edge component) to the common edge detection unit 321 as a first edge signal.
The second edge detection unit 320 extracts a high frequency component (edge component) from the image signal input from the frame delay unit 212 (image signal one subframe before the image signal input from the frame converter unit 101), The two edge signals are output to the common edge detection unit 321. The processing contents of the first edge detection unit 319 and the second edge detection unit 320 may be the same or different.
The common edge detection unit 321 compares the first edge signal generated by the first edge detection unit 319 with the second edge signal generated by the second edge detection unit 320. Then, the common edge detection unit 321 detects pixels having the same sign and both signal values having a predetermined value or more as common edge pixels.
The stationary pixel adjacent edge detection unit 322 detects a common edge pixel adjacent to the stationary pixel detected by the stationary pixel detection unit 318 as a stationary edge pixel, and outputs the detection result as a stationary edge detection signal.

  The tap control LPF unit 314 performs suppression processing on the frame difference signal based on the stationary edge detection signal, and outputs the result to the suppression region determination unit 215 as a high frequency suppression frame difference signal. Specifically, for each pixel, the tap control LPF unit 314 suppresses the frame difference signal of the pixel by excluding the influence of the pixel located outside the pixel determined to be a stationary edge with respect to the pixel. Apply.

The tap control LPF unit 314 can be realized by a filter as shown in FIG. 11, for example.
In FIG. 11, d (x) is the value of the frame difference signal of the pixel x, and e (x) is the value of the still edge detection signal of the pixel x. A pixel with x = 0 is a pixel to be processed, and a pixel with x = n (n = −2, −1, 1, 2) is an n number of pixels with x = 0. Specifically, a pixel with x = −1 is a pixel located on a predetermined direction side (for example, the left side) by one pixel with respect to a pixel with x = 0, and a pixel with x = −2 has a pixel with x = 0. This is a pixel located on the predetermined direction side by two pixels with respect to the pixel. The pixel of x = 1 is a pixel located on the opposite side (for example, the right side) of the predetermined direction by one pixel with respect to the pixel of x = 0, and the pixel of x = -2 is 2 for the pixel of x = 0. It is a pixel located on the opposite side of the predetermined direction by the amount of pixels. C (0), C (1), and C (2) are filter coefficients (filter tap coefficients). m (x) is a value obtained by multiplying (weighting) the value of the frame difference signal of the pixel x by the filter coefficient. lpf (0) is an output value of the filter. In this embodiment, the stationary edge detection signal is a signal that is 1 for pixels of the stationary edge and 0 for pixels other than the pixels of the stationary edge.
In this embodiment, the frame difference signal and the stationary edge detection signal are input in order for each pixel. For example, for each line of the image, signals from the leftmost pixel of the image are input in order. Therefore, a signal of a desired pixel can be taken out by using a delay circuit (“delay” in the figure).

The tap control LPF unit 314 controls the filter coefficient based on the stationary edge detection signal, and performs filter processing for each pixel. Specifically, the tap control LPF unit 314 sets the filter coefficient for multiplying (weighting) the frame difference signal of the pixel located outside the pixel determined to be a stationary edge with respect to the pixel to be processed to 0. Thereby, the influence of the pixel located outside the pixel determined as the still edge with respect to the pixel to be processed is excluded. For example, in FIG. 11, when the value of e (1) is 1 and the values of e (2), e (0), e (-1), and e (-2) are 0, d (2) And d (1) are weighted with a filter coefficient of 0, and the output value of the filter is the value obtained by Equation 1.
Output value = c (0) × d (0) + c (1) × d (−1) + c (2) × d (−2)
... (Formula 1)

  In this way, the value of the pixel located outside the pixel determined to be a stationary edge with respect to the pixel to be processed (pixels d (1) and d (2) in the above example) is excluded, The output value of the filter is calculated. Therefore, it is possible to prevent the value of a pixel that is not a still pixel from affecting the output value of the filter in the filter process for the still pixel. As a result, it is possible to suppress the area of the still pixel as a suppression area. Specifically, since the value of the frame difference signal of the still pixel is small, the output value of the filter also becomes a small value, and the region on the still pixel side with the still edge as a boundary can be set as a non-suppression region.

  Although FIG. 11 shows an example in which the tap control LPF unit 314 is a one-dimensional filter in the horizontal direction (left-right direction), the configuration of the tap control LPF unit 314 is not limited to this. The tap control LPF unit 314 may be a one-dimensional filter in a vertical direction (up and down direction) or an oblique direction, or may be a two-dimensional filter.

FIG. 12 is a diagram illustrating an example of a flow of a determination process for determining whether or not the area is a suppression region according to the third embodiment.
First, for each pixel, the absolute value of the difference between the image signal of the previous subframe and the image signal of the current subframe is calculated. Also, edge pixels are detected from each of the previous subframe image signal and the current subframe image signal.
Then, a pixel that is an edge and is adjacent to a pixel having a small absolute value of the difference is detected as a still edge pixel.
Next, based on the detection result of the pixels of the stationary edge, the absolute value of the difference is filtered.
And the suppression area | region (and non-inhibition area | region) is detected by comparing the result of the said filter process, and a determination threshold value.
As described above, the region on the still pixel side with the still edge as a boundary is set as a non-suppression region. Therefore, the still pixel region is set as a suppression region, and image distortion due to suppression of variation in the still pixel region can be suppressed.

FIG. 13 shows an example of a comparison between the image processing according to the first embodiment and the image processing according to the third embodiment with respect to a region that moves between frames. In FIG. 13, the horizontal axis represents the pixel position, and the vertical axis represents the pixel value. FIG. 13 shows the state of image processing for a signal in an image area that moves to the right (left as viewed from the viewer) from the previous frame to the current frame, as in FIG.
In the image processing of the first embodiment, the compensation value is generated using the time direction smoothed image signal. Specifically, when the enhancement process and the restriction process are sequentially performed on the time-direction smoothed image signal, the change amount of the pixel value by the restriction process is set as the compensation value.

Also in the image processing of the third embodiment, the compensation value is generated using the time direction smoothed image signal. However, in the third embodiment, a region with motion is detected, and among the generated compensation values, the compensation value added to the signal of the detected region (region with motion) is reduced (specifically, 0). For this reason, in the third embodiment, the compensation of the amount of change is suppressed in the region where there is motion, and the high-frequency-suppressed image signal in which the spatial high-frequency component in the region where there is motion is suppressed more than in the first embodiment. can get. As a result, it is possible to suppress the double image from being visually recognized in the region where the pixel value has changed from the previous subframe, that is, the region with movement, compared to the first embodiment, and the moving image visibility is improved.
Even in the image processing according to the third embodiment, the compensation value added to the signal of the non-motion region (non-suppression region) is not reduced. That is, in the third embodiment, the same processing as that for the non-motion area in the first embodiment is performed. As a result, in a region where there is no motion, the same signal as that obtained by conventional processing is obtained as the high-frequency suppression image signal and high-frequency emphasized image signal output to the display device. Therefore, the average image signal of the high-frequency emphasized image signal and the high-frequency suppressed image signal is made equal to the input image signal as before. As a result, a still image that is equal to the image based on the input image signal and has no distortion can be displayed.

  As described above, according to the present embodiment, when generating the high frequency suppressed image signal to be output to the display device, the compensation value is generated using the time direction smoothed image signal. Then, among the generated compensation values, the compensation value to be added to the signal in the region with motion is reduced. Thereby, even when the pixel value after the high frequency component is emphasized becomes a value outside the specified range, the moving image visibility can be sufficiently improved. Specifically, the moving image visibility can be improved as compared with the first embodiment.

In the present embodiment, among the compensation values generated by the limit unit 106, the compensation value added to the suppression region is set to 0. However, the present invention is not limited to this. Of the compensation values generated by the limit unit 106, the compensation value to be added to the suppression region may be reduced. The amount of reduction of the compensation value may be increased from the boundary between the non-suppression region and the suppression region toward the inside of the suppression region.
Note that the method for detecting a region with movement according to the third embodiment may be applied to the second embodiment, and the method for detecting a region with movement according to the second embodiment may be applied to the third embodiment.
Each function described in the first to third embodiments may be realized by hardware or may be realized by software.

DESCRIPTION OF SYMBOLS 101 Frame converter part 102 Time direction filter part 104 High frequency suppression part 105 High frequency emphasis part 106 Limit part 107 Adder part 108,208 2nd selection part 211,311 Compensation suppression control part

Claims (18)

Frame rate conversion means for generating a plurality of subframes from a frame of the input image signal and increasing the frame rate of the input image signal;
An emphasis unit that generates a high-frequency emphasized image signal by performing an emphasis process that emphasizes a spatial high-frequency component on the image signal of the subframe;
Suppression means for generating a high-frequency-suppressed image signal by applying a suppression process for suppressing a spatial high-frequency component to the image signal of the subframe;
The high-frequency emphasized image signal is subjected to a restriction process that restricts a pixel value to a value within a specified range, and a compensation value that compensates for the amount of change in the pixel value due to the restriction process is added to the high-frequency suppression image signal. Adjusting means for performing compensation processing;
An output means for alternately outputting the high-frequency emphasized image signal subjected to the restriction process and the high-frequency suppressed image signal subjected to the compensation process to the display device every subframe;
Have
The adjusting unit generates the compensation value so that a compensation value to be added to a signal in a moving region of the high-frequency-suppressed image signal is smaller than a change amount of a pixel value due to the restriction process. A featured image processing apparatus. The adjustment means uses the time-direction smoothed image signal, which is an image signal generated by performing a smoothing process to reduce the amount of change in the pixel value between subframes on the subframe image signal, and calculates a compensation value. The image processing apparatus according to claim 1, wherein the image processing apparatus generates the image processing apparatus. 3. The adjustment unit according to claim 2, wherein when the enhancement process and the restriction process are sequentially performed on the time-direction smoothed image signal, a change amount of a pixel value due to the restriction process is used as a compensation value. An image processing apparatus according to 1. It further has a motion detection means for detecting a motion region of the image signal of the subframe,
4. The image processing apparatus according to claim 2, wherein the adjustment unit reduces a compensation value to be added to a motion area signal detected by the detection unit among the generated compensation values. 5. . 5. The image processing apparatus according to claim 4, wherein the adjustment unit does not reduce a compensation value to be added to a signal in a non-motion region, which is a region other than the motion region detected by the detection unit. . It further has a motion detection means for detecting a motion region of the image signal of the subframe,
The adjustment means uses the amount of change in the pixel value due to the restriction process as a compensation value, and among the compensation values, reduces the compensation value added to the signal of the region with motion detected by the detection means. The image processing apparatus according to claim 1. The image processing apparatus according to claim 6, wherein the adjustment unit does not reduce a compensation value to be added to a signal of a non-motion region that is a region other than the motion region detected by the detection unit. . 8. The adjustment unit according to any one of claims 4 to 7, wherein the adjustment unit sets a compensation value to be added to a signal of a motion area detected by the detection unit out of the generated compensation values. An image processing apparatus according to 1. A frame rate conversion step of generating a plurality of subframes from a frame of the input image signal and increasing a frame rate of the input image signal;
An emphasis step for generating a high-frequency emphasized image signal by applying an emphasis process that emphasizes a spatial high-frequency component to the image signal of the subframe;
A suppression step of generating a high-frequency-suppressed image signal by applying a suppression process that suppresses a spatial high-frequency component to the image signal of the subframe;
The high-frequency emphasized image signal is subjected to a restriction process that restricts the pixel value to a value within a specified range, and a compensation value that compensates for the amount of change in the pixel value due to the restriction process is added to the high-frequency-suppressed image signal. An adjustment step for performing compensation processing;
An output step of alternately outputting the high-frequency emphasized image signal subjected to the restriction processing and the high-frequency suppression image signal subjected to the compensation processing to the display device every subframe;
Have
In the adjustment step, generating a compensation value so that a compensation value to be added to a signal in a moving region of the high-frequency-suppressed image signal is smaller than a change amount of a pixel value due to the restriction process. A control method for an image processing apparatus. In the adjustment step, a compensation value is calculated using a time-direction smoothed image signal that is an image signal generated by performing a smoothing process that reduces the amount of change in pixel value between subframes on the image signal of the subframe. The method according to claim 9, wherein the image processing apparatus is generated. 11. The adjustment step, wherein when the enhancement process and the restriction process are sequentially performed on the time-direction smoothed image signal, a change amount of a pixel value due to the restriction process is set as a compensation value. A control method for the image processing apparatus according to claim 1. A motion detection step of detecting a motion region of the image signal of the subframe;
The image processing apparatus according to claim 10 or 11, wherein, in the adjustment step, a compensation value to be added to a signal of a motion area detected in the detection step is reduced among the generated compensation values. Control method. 13. The image processing apparatus according to claim 12, wherein the adjustment step does not reduce a compensation value to be added to a signal in a non-motion region that is a region other than the motion region detected in the detection step. Control method. A motion detection step of detecting a motion region of the image signal of the subframe;
In the adjustment step, the amount of change in the pixel value due to the restriction process is used as a compensation value, and the compensation value to be added to the motion region signal detected in the detection step is reduced in the compensation value. A method for controlling an image processing apparatus according to claim 9. The image processing apparatus according to claim 14, wherein the adjustment step does not reduce a compensation value to be added to a signal in a non-motion region, which is a region other than the motion region detected in the detection step. Control method. 16. The adjustment step, wherein a compensation value to be added to a motion area signal detected in the detection step is set to 0 among the generated compensation values. A control method for the image processing apparatus according to claim 1. Frame rate conversion means for generating first and second sub-frames from a frame of the input image signal and converting a frame rate of the input image signal;
First generation means for generating a high-frequency image signal having more spatial high-frequency components than low-frequency components from the image signal of the first subframe;
Second generation means for generating, from the image signal of the second subframe, a low-frequency image signal in which spatial low-frequency components are larger than high-frequency components;
Compensation processing for limiting the pixel value to a value within a predetermined range for the high-frequency image signal, and adding a compensation value for compensating for the amount of change in the pixel value due to the limitation processing to the low-frequency image signal. Adjusting means to be applied;
An output means for alternately outputting the high-frequency image signal subjected to the restriction processing and the low-frequency image signal subjected to the compensation processing to a display device every subframe;
Have
The adjusting unit generates the compensation value so that a compensation value to be added to a signal in a moving region of the low-frequency image signal is smaller than a change amount of a pixel value by the restriction process. An image processing apparatus. A frame rate conversion step of generating first and second sub-frames from a frame of the input image signal and converting a frame rate of the input image signal;
A first generation step of generating a high-frequency image signal having more spatial high-frequency components than low-frequency components from the image signal of the first subframe;
A second generation step of generating, from the image signal of the second subframe, a low-frequency image signal in which spatial low-frequency components are larger than high-frequency components;
Compensation processing for limiting the pixel value to a value within a predetermined range for the high-frequency image signal, and adding a compensation value for compensating for the amount of change in the pixel value due to the limitation processing to the low-frequency image signal. Adjustment steps to be applied;
An output step of alternately outputting the high-frequency image signal subjected to the restriction process and the low-frequency image signal subjected to the compensation process to a display device every subframe;
Have
In the adjustment step, the compensation value is generated so that a compensation value to be added to a signal in a moving region of the low-frequency image signal is smaller than a change amount of a pixel value by the restriction process. A control method of the image processing apparatus.

Images