JP2006121274A - Scene change detecting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a scene change detecting device capable of making natural scene change matching characteristics of the human eye. <P>SOLUTION: The scene change detecting device comprises luminance average measuring portions 1 and 2 which find average values, histogram extraction portions 3 and 6 which represent distributions of luminance in histograms, a luminance shift circuit portion 20 which equalizes the average value of luminance level of the whole 1st image and the average value of luminance level of the whole 2nd image calculated by the luminance average value measuring portions to the same luminance level and shifts the distribution of luminance within a designated reference range represented in the histograms by the histogram extraction portions 3 and 6, a difference arithmetic circuit portion 21 which finds difference values between histogram values of the 1st image or 2nd image shifted by the luminance shift circuit 20, and a judgment circuit portion which judges whether or not the difference values found by the difference arithmetic circuit portion 21 are larger than a designated value. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a scene change detection device that detects switching of images from time-sequential frame images representing moving images.

  In a projection projector or the like that inputs an image from a TV broadcast, a high-definition broadcast, a VTR, a DVD, etc. and projects a large screen on the screen, an image of a sudden change in the screen without a continuity (referred to as a scene change) is input. On a bright screen where power consumption increases, the power is momentarily insufficient, and a bright screen is projected darker.On the other hand, on a dark screen where power consumption decreases, the power is momentarily excessive and the dark screen is brighter. Projection has caused abnormalities on the screen.

  For example, as shown in FIG. 7A, when the input image suddenly changes from a dark image to a bright image and then changes from a bright image to a dark image, the brightness of the image projected on the screen is as follows. Less, it gradually brightens from the next frame and finally gets a normal brightness at the end of the fourth frame, and then suddenly changes to a dark image, the first frame gradually darkens and then recovers to a normal image Therefore, the projected image was deteriorated.

  And the input image as described above suddenly changes from a dark image to a bright image, or suddenly changes from a bright image to a dark image. Conventionally, as shown in FIG. 8, there is a method of extracting a switching portion of an image to be used as a scene change by a scene change detection device 11, and it is considered to control an image projected by this scene change. It was.

  The scene change detection device 11 shown in FIG. 8 includes, for example, a one-frame delay device 12, an inter-frame difference calculation device 13, and a scene change determination device 14. The scene change determination device 14 makes one frame for the input image Vin. The determination is based on the sum of absolute values in difference data Df0 from the delayed image Vm. That is, if the sum of the absolute values of the difference data Df0 is larger than a predetermined threshold value, the scene change determination device 14 determines that there is a scene change and sets the scene change flag Db0 to ON.

As an application in the case of handling a scene change, Patent Document 1 describes as follows that extracts a still image when image compression is performed.
1. Special effects such as scene switching and fade-in / fade-out in the video are automatically detected. These characteristic scene changes are detected by scene changes, and the video content is reduced and displayed as a still image. Make a grasp.
2. With regard to a noise reduction device for reducing noise in a video signal, for example, noise reduction processing is performed when a scene change is detected using image field correlation or frame correlation.
3. Keeps the frame when changing scenes clean (not affected by the interpolation process). When a scene change is detected, the interpolation processing of the immediately following frame is reduced from 4-field processing to 2-field processing, and interpolation processing between images without frame correlation is prevented.
4). When encoding, pre-encode the beginning of the scene detected by “automatic scene change detection” and then perform the actual encoding. This gives an accurate relationship between distortion and bit rate.
The scene change used for still image extraction when performing this image compression is performed without lowering the compression rate for a dozen or so frames of moving images, and is performed on a screen that controls an image that changes in real time within one frame. It has been difficult to use in a projection type projector that projects on a large screen.
JP-A-6-22298

  However, the scene change determination performed based on the sum of absolute values in the difference between the input image Vin and the image Vm delayed by one frame as described above must be performed by calculating the difference data of the entire one frame image. It takes more than one frame to make a change determination, and there is a problem that a malfunction is likely to occur due to a change in the high-frequency component of the entire screen or a partial change in light and dark. There is a problem that it is difficult to use it for a projection type projector that projects a large screen in real time.

  Therefore, the present invention has been made to solve the above-described problems, and there are few malfunctions even with respect to a frame image having a change in high-frequency components of the entire screen or a partial change in light and dark. An object of the present invention is to provide a scene change detection device capable of accurately determining a scene change within a time of one frame.

  As a means for achieving the above object, the scene change detection device according to the first aspect of the present invention, when performing image display using a liquid crystal display element, causes the image input to the liquid crystal display element to change from the first image to the second In the scene change detection device that outputs a change control signal to the liquid crystal display element when the image is changed, a luminance average measuring unit that calculates an average value after measuring the luminance level of the entire first and second images; A histogram extracting unit that histograms each luminance distribution within a predetermined reference range from the luminance levels of the entire first and second images measured by the luminance average measuring unit; and the first and second images A difference value between the average values of the luminance levels of the entire image is obtained, and based on the difference value, the average value of the luminance levels of the entire first image calculated by the luminance average value measuring unit and the entire second image The brightness level of the first image or the second image within the predetermined reference range that has been histogrammed by the histogram extraction unit, and the average value of the brightness levels of the first and second images coincides with each other. A shift circuit unit, a difference calculation circuit unit for obtaining a difference value between the histogram values of the first image or the second image within the predetermined reference range shifted by the luminance shift circuit unit, and the difference calculation A determination circuit unit that determines whether a difference value between the histogram values obtained by the circuit is equal to or greater than a predetermined value; and if the determination circuit unit exceeds the predetermined value, the first image and When a signal with a change is output to the image between the second image and the value is equal to or less than the predetermined value, the image between the first image and the second image is not changed. A signal output circuit unit for outputting a signal, and a signal processing circuit unit for outputting the change control signal for expanding a dynamic lens of a response characteristic of the liquid crystal display element based on a signal output from the signal output circuit unit. The present invention provides a scene change detection device characterized by

  According to the present invention, scene change determination according to the brightness of an image is performed using an APL (Average Picture Level) between an input image and an image delayed by one frame and a difference value based on a partial histogram. As a result, there are few malfunctions with respect to a frame image having a change in high-frequency components or a partial change in brightness, and an accurate scene change can be performed within a processing time of one frame or less.

A scene change detection apparatus according to an embodiment of the present invention will be described below with reference to FIGS.
FIG. 1 is a diagram showing a configuration of a scene change detection apparatus according to an embodiment of the present invention. FIG. 2 is an explanatory diagram of a histogram and APL used for scene change determination. FIG. (B) is a figure which shows the histogram and APL of an image with large APL. FIG. 3 is a diagram showing the difference between the histogram and the APL when the screen changes from a dark screen to a bright screen. FIG. 4 is a diagram showing a table used for scene change determination. FIG. 5 is a diagram showing gamma conversion characteristics. 6A and 6B are diagrams showing the luminance level of the projected image luminance with respect to the input image level. FIG. 6A is a diagram showing the luminance level when the change in the input image level is a straight line. FIG. It is a figure which shows the luminance level in the case of B. FIG. 7 is a diagram showing generation of a projected image with little image quality degradation using a scene change, (A) is a diagram showing an image level before processing, and (B) is a diagram showing an image level after processing. is there. (C) is a diagram showing a configuration of an apparatus that generates a projection image with little image quality degradation.

  As shown in FIG. 1, in the scene change detection apparatus according to the embodiment of the present invention, when an image is displayed using a liquid crystal display element, the image input to the liquid crystal display element is changed from the first image to the second image. In the scene change detection device that outputs a change control signal to the liquid crystal display element, after measuring the luminance levels of the entire first and second images, the luminance average measurement units 1 and 2 (APL detection unit) for obtaining an average value 1, 2), and histogram extraction units 3 and 6 that histogram each luminance distribution within a predetermined reference range from the luminance levels of the entire first and second images measured by the luminance average measuring unit, A difference value between the average values of the luminance levels of the entire first and second images is obtained, and based on the difference value, the average value of the luminance levels of the entire first image calculated by the luminance average value measuring unit and the second value are calculated. Brightness level of the entire image And a luminance shift circuit unit 20 that shifts the luminance distribution of the first image or the second image within the predetermined reference range histogrammed by the histogram extraction units 3 and 6. And the difference calculation circuit unit 21 for calculating the difference value between the histogram values of the first image or the second image within the predetermined reference range shifted by the luminance shift circuit unit 20, and the difference calculation circuit 21. A determination circuit unit 22 that determines whether or not the difference value between the histogram values is equal to or greater than a predetermined value, and if the determination circuit unit 22 exceeds the predetermined value, the interval between the first image and the second image A signal output circuit unit 23 that outputs a signal with no change to an image between the first image and the second image when a signal with a change is output to the image of Signal output circuit 3 based on the signal output from the one in which a signal processing circuit section 24 for outputting a change control signal to widen the dynamic lens response characteristics of the liquid crystal display device, is constituted by the scene change determination unit 8 comprising a.

  Next, an operation example will be described. First, in FIG. 1, the input image Vin is added to the APL detection unit 1. The APL detection unit 1 adds and averages the input image Vin in units of pixels to detect the entire average value level Ap0. The average value level Ap0 is added to the histogram detection range setting unit 2, and the histogram detection range Hw0 is set and added to the histogram extraction unit 3. Next, the input image Vin is input to the histogram extraction unit 3, and the histogram Hg 0 of the histogram detection range Hw 0 is extracted and added to the scene change determination unit 8.

Then, an image Vm delayed by one frame obtained by storing the input image Vin in the frame memory 4 is added to the APL detection unit 5. The APL detection unit 5 averages the images Vm in units of pixels and detects the overall average value level Ap1. Then, the average value level Ap1 is added to the histogram detection range setting unit 6, and the histogram detection range Hw1 is set and added to the histogram extraction unit 7. Next, the image Vm is input to the histogram extraction unit 7, and the histogram Hg 1 of the histogram detection range Hw 1 is extracted and added to the scene change determination unit 8.
A configuration example of the scene change circuit 8 is shown in the lower part of FIG. 1 as a luminance shift circuit unit 20, a difference calculation circuit unit 21, a determination circuit unit 22, a signal output circuit unit 23, and a signal processing circuit unit 24.

The histogram detection range setting units 2 and 6 set the histogram detection range according to the APL level as shown in FIG.
For example, the histogram is set to 16 levels with respect to the maximum signal level, and the setting range is set to 6 levels centering on APL.

  Then, as shown in FIG. 2A, when the APL level is located at the fifth level, a histogram at the third to eighth levels centering on the APL level Ap0 is set as the detection range A. Then, for example, as shown in FIG. 2B, when the APL level is located at the eleventh level, the 9th to 14th level histogram centering on the APL level Ap0 is set as the detection range B. The detection range A and the detection range B are set to a constant range that does not change depending on the position of the APL level. However, the detection range is narrowed when the APL level is low in proportion to the APL level, and the detection range is widened when the APL level is high. You may set to do.

  The detection range A is output from the histogram detection range setting unit 2 as a control signal Hw0 and added to the histogram extraction unit 3. The detection range B is output from the histogram detection range setting unit 6 as a control signal Hw 1 and added to the histogram extraction unit 7.

On the other hand, the input image Vin is also added to the histogram extraction unit 3 to extract a histogram from the current image shown in FIGS. 2 (A) and 2 (B). In the histogram extraction unit 3, the histogram Hg0 of only the detection range A is extracted by the control signal Hw0 when the histogram is extracted, and is sent to the scene change determination unit 8.
Then, the image Vm delayed by one frame is added to the histogram extraction unit 7 to extract a histogram from the image one frame before shown in FIG. In the histogram extraction unit 7, the histogram Hg 1 of only the detection range B is extracted by the control signal Hw 1 at the time of histogram extraction and sent to the scene change determination unit 8.

Then, the absolute value difference level Hn at each stage n is calculated by the scene change determination unit 8 according to the histogram in the detection range.
Absolute value difference data Hn = ‖Hg0n−Hg1n‖. . . (1) Formula

The scene change determination unit 8 inputs APL and histogram, respectively, and determines the scene change.
Scene change conditions (conditions for turning on scene change with Db0 = 1):
[1] Fluctuation of APL (1) Dark → Bright (AP0-AP1)> K0 × APmax. . . (2) Formula
K0: constant, APmax: APL maximum value (2) Bright → dark (AP1-AP0)> K1 × APmax. . . (3) Formula
K1: Constant Since the human eye reacts quickly to light and reacts slowly to darkness, the constant is set so that K0 <K1 according to this reaction. For example, K0 = 0.2 and K1 = 0.4.
[2] Histogram fluctuation When the APL fluctuation is small, the sum of the absolute value difference levels Hn calculated by the expression (1) in the respective detection ranges as shown in (3) of FIGS. judge.
ΣHn> K2 × Hmax. . . (3) Formula
K2: constant, Hmax: maximum value of detection range histogram (in FIG. 2A, n = 3 to 8, and in FIG. 2B, n = 9 to 14. At this time, K2 is 0 by experiment. .. set to a range of 3 to 0.5.)
[3] Histogram fluctuation considering APL fluctuation Since the human eye reacts quickly to brightness and slowly to darkness, Ap output from the APL selection section 3 (APL with a maximum value of 1) The constant K2 in the equation (3) is set as K3 according to the level of the level) as follows.
K3 = K2 × (1-Ap)
That is, the equation (3) is ΣHn> K3 × Hmax. . . (4)
Next, the case where the APL level shown in FIG. 2A is low is set as a determination value B from the equation (4), and the case where the APL level shown in FIG. 2B is higher than that in FIG. Assuming that the determination value A is used, the determination value A is smaller than the determination value B. As a result, the reaction to the scene change is accelerated when it is bright.

If the detection range of the histogram is limited to a partial range centered on the APL level in this way, the entire screen is determined at the APL level, and when the fluctuation of the APL level is small, the fluctuation of the histogram is partial. By making a determination based on this, it is possible to accurately detect a scene change.
Further, since the detection range of the histogram has 16 signal levels and 6 levels of approximately 1 / 2.7, the processing time can be shortened to approximately 1 / 2.7, so that the actual time of one frame image can be reduced. Is enough to make scene change judgments using histograms.

As described above, the scene change is determined based on the fluctuation of the histogram centering on APL.
For example, if the dark image Vin shown in FIG. 3 (1) has a low APL and becomes a bright image Vm shown in FIG. 3 (2), the APL level changes but the histogram does not change, so the scene change flag is OFF. It is.
This state corresponds to the case where the sun is actually covered with clouds or the illumination light fluctuates, and the content of the image does not change, so the continuity of the image can be maintained by turning off the scene change flag. I can do it.

  That is, as shown in FIG. 3 (3), according to the absolute value of the difference (Vin−Vm) of the histogram of the entire image, a large difference occurs in the histogram, so the scene change flag must be turned on. 4) According to the partial difference (Vin− {Vm− (Ap1−Ap0)}) of the histogram obtained by subtracting the APL difference (AP1−Ap0) in advance, no difference occurs in the histogram. Is OFF.

FIG. 4 shows a table used for the scene change determination unit 9. A scene change is determined based on this table.
In the table, when Ap0-Ap1 is <-0.2, the dark screen is changed to the bright screen, and the light adaptation is performed. Therefore, the scene change is turned ON while the difference between the histograms is small.
When the brightness of the screen corresponding to a range of Ap0-Ap1 in the range of -0.2 to +0.2 does not vary much, the difference between the histograms becomes slightly larger before the scene change is turned on. This is because there are few scene changes at the same brightness.
When Ap0-Ap1 is +0.2 <, the screen changes from a bright screen to a dark screen and dark adaptation is performed. Therefore, the scene change is turned on by slightly increasing the histogram difference compared to the light adaptation.

  In order to obtain a projection image to be used for a projection type projector or the like using this scene change, first, when the scene change is turned on, the linearity of the input image is set to gamma A as shown in FIG. This is converted to gamma B to obtain a projection image of the first frame.

Next, gamma conversion of the projected image when the scene change is turned on will be described with reference to FIG.
First, FIG. 6A shows the luminance level in the projected image luminance when the input image level changes from 0 to 100%. Since the luminance level is proportional to the input image level in the central proportional area P1-P2, it is used as it is.

  Next, when the input image level changes linearly from P2 to the brightest 100% level as indicated by a dotted line, the luminance level at this time changes between P2 and P3, and becomes an insufficient region for the increase in luminance level.

  As shown in the lower part of the graph, when the input image level changes linearly from P1 to the darkest 0% level as indicated by a dotted line, the luminance level at this time is a change between P1 and P0. This is a shortage area.

  Therefore, when the input level changes from the darkest 0% level to the brightest 100% level, the luminance level becomes the luminance display range (A), which is displayed because the maximum brightness of the luminance level of the projected image decreases and the minimum dark portion increases. The brightness range becomes very narrow.

  Therefore, the dotted line portion of the input image of FIG. 6A is converted into gamma B of FIG. 5 to obtain the dotted line portions of FIG. 6B P3-P5 and FIG. 6B P1-P4. The luminance display range (B) of the solid line portion in FIG. 6 (B) equivalent to the input image level in the dotted line portion of 6 (A) can be obtained.

  Then, as shown in FIG. 7A, for example, as shown in the dotted line portion of the input image, the first frame has a low input image level, and the next first to fifth frames have a 100% input image level. Assuming that the low input image level is from the 9th frame to the 9th frame, the projected image brightness is the brightness level of point P2 in FIG. 6A in the 1st frame, and then the brightness level is increased for each frame and is 100 in the 5th frame. % Luminance level.

Next, when the input image level of the fifth frame is changed from the 100% input image level to the low input image level of the sixth frame, the projected image luminance becomes the luminance level of point P1 in FIG. The level is lowered and becomes a luminance level equivalent to the low input image level at the ninth frame.
If the linearity of the input image level and the brightness level of the projected image is different from each other in this way, the image level with respect to the elapsed time (frame) is shifted, resulting in degradation of image quality.

Therefore, as shown in FIG. 7C, the scene change detection device 9 and the gamma conversion device 15 use the scene change to prevent the image quality from being deteriorated.
First, the first frame has a low input image level and the input image Vin is set to a 100% input image level from the first frame to the fifth frame and a low input image level from the sixth frame to the ninth frame. Input to the detection device 9. Then, since the scene change is determined in the first frame, the scene change flag Db0 at this time is turned ON and input to the gamma conversion device 15. A 1-frame delayed image Vm of the input image is also output from the scene change detection device 9, and the linearity of the input image of the first frame is set to gamma B by the gamma conversion device to obtain a projection image Vp.

Then, as shown in FIG. 7B, the gamma conversion apparatus 10 obtains a projection image luminance close to a luminance level of almost 100% by the projection image Vp subjected to the gamma conversion as shown in FIG. 7B.
Next, although the scene change flag Db0 is turned OFF, the gamma conversion apparatus 10 continues to perform gamma conversion according to the luminance level with respect to the input image of the projected image luminance in FIG. 6 (A). Assuming that the third frame is between gamma A and gamma B and the fourth frame is gamma A, a luminance level substantially equal to the input image level is obtained as shown in FIG.

  When the image input level changes from the 100% input image level to the low input image level from the 5th frame to the 6th frame, a scene change is detected and the scene change flag Db0 is turned ON. If the gamma A is changed to gamma B, the next 8th frame is between gamma A and gamma B, and the 9th frame is gamma A, the luminance level is almost equal to the input image level as shown in FIG. Is obtained.

  Thus, by changing the linearity of the projected image from gamma A to gamma B at the time of a scene change, an input screen for an abrupt change indicated by a dotted line as shown in FIG. 7 (A) is shown in FIG. 7 (B). Such a natural change image can be projected on a screen by a projection projector or the like.

  As described above, the scene change detection apparatus according to the embodiment of the present invention, when detecting a scene change between an image one frame before and the current image, not a whole histogram but a part of the histogram according to APL. By using as the scene change detection range, the calculation time of the histogram can be reduced to approximately 1 / 2.7 or less of one frame, so that the image can be continuously processed in real time.

  In addition, a part of the histogram based on the APL based on the DC component obtained by averaging the high-frequency component and the frame image having a change in high-frequency component that is easy to recognize as a scene change and a partial change in light and dark By using this, it is possible to eliminate these two effects, and it is possible to perform an accurate scene change with few malfunctions.

  As a detection condition, for example, for scene change detection from a bright scene, the absolute value difference of the range on the upper side of the histogram exceeds the table value shown on the left side of FIG. When the absolute value difference of the range on the lower side exceeds the table value on the right side of FIG. 4, a scene that matches the characteristics of the human eye reacts quickly to brightness and reacts slowly to darkness Change can be judged.

It is a figure which shows the structure of the scene change detection apparatus in embodiment of this invention. (A) is a diagram showing a histogram and APL of an image having a low APL, and (B) is a diagram showing a histogram and APL of an image having a large APL. It is a figure which shows the difference of a histogram and APL at the time of changing from a dark screen to a bright screen. It is a figure which shows the table used for determination of a scene change. It is a figure which shows a gamma conversion characteristic. It is a figure which shows the luminance level of the projection image brightness | luminance with respect to an input image level, (A) is a figure which shows a luminance level when an input image level change is made into a straight line, (B) is an input image level made into gamma B It is a figure which shows the luminance level in a case. It is a figure which shows the production | generation of a projection image with little image quality degradation using a scene change, (A) is a figure which shows the image level before a process, (B) is a figure which shows the image level after a process, C) is a diagram showing a configuration of an apparatus that generates a projection image with little image quality degradation. It is a figure which shows the structure of the conventional scene change detection apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... APL detection part, 2 ... Histogram detection range setting part, 3 ... Histogram extraction part, 4 ... Frame memory, 5 ... APL detection part, 6 ... Histogram detection range setting part , 7 ... Histogram extraction unit, 8 ... Scene change determination unit, 9 ... Scene change detection device, 10 ... Gamma conversion device, 11 ... Scene change device, 12 ... 1 frame delay Device: 13 ... Inter-frame difference detection device, 14 ... Scene change determination device, 20 ... Luminance shift circuit unit, 21 ... Difference calculation circuit unit, 22 ... Judgment circuit unit, 23 ... .Signal output circuit section, 24... Signal processing circuit section

Claims (1)

A scene change that outputs a change control signal to the liquid crystal display element when an image input to the liquid crystal display element is changed from a first image to a second image when an image is displayed by a projector using the liquid crystal display element. In the detection device,
A luminance average measuring unit for obtaining an average value after measuring the luminance levels of the entire first and second images;
A histogram extracting unit that histograms each luminance distribution within a predetermined reference range from the luminance levels of the entire first and second images measured by the luminance average measuring unit;
A difference value between the average values of the luminance levels of the first and second images is obtained, and an average value of the luminance levels of the entire first image calculated by the luminance average value measurement unit is calculated based on the difference value. The luminance value of the first image or the second image within the predetermined reference range that is histogrammed by the histogram extraction unit is matched with the average value of the luminance levels of the entire second image. A luminance shift circuit unit for shifting the distribution of
A difference calculation circuit unit for calculating a difference value between the histogram values of the first image or the second image within the predetermined reference range shifted by the luminance shift circuit unit;
A determination circuit unit that determines whether or not a difference value between the histogram values obtained by the difference calculation circuit is a predetermined value or more;
When the predetermined value is exceeded by the determination circuit unit, a change signal is output to the image between the first image and the second image. A signal output circuit unit that outputs a signal without change to an image between the first image and the second image;
Based on a signal output from the signal output circuit unit, a signal processing circuit unit that outputs the change control signal that expands a dynamic lens of a response characteristic of the liquid crystal display element;
A scene change detection device comprising:

Images