JP2010103777A - Moving image processing apparatus, moving image processing method, program thereof, recording medium and moving image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image processing apparatus, a moving image processing method, a program thereof, a recording medium and a moving image display device, which is applied even when it is difficult to calculate an appropriate correction parameter only from histogram. <P>SOLUTION: The moving image processing apparatus 10 has: a recording unit 106 which records a plurality of scenes by associating main histogram of the scenes with correction parameters suitable for the scenes; a histogram calculation unit 101 which calculates the histogram from pixel values of frames; a correlation degree calculation unit 102 which calculates correlation degrees between the calculated histogram and the histogram of the plurality of recorded scenes; a correction parameter calculation unit 103 which calculates the correction parameters; and a correction processing unit 105 which performs correction. The correlation parameter calculation unit 102 calculates the correction parameters suitable for the frames using the correction parameters recorded according to the scenes based on the correlation degrees between the frames and the scenes recorded in the recording unit 106 in advance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving image processing apparatus, a moving image processing method, a program thereof, a recording medium, and a moving image display apparatus that correct a moving image.

  As a technique for correcting a moving image such as a natural image, CG, or animation, a technique for calculating an image correction parameter from each frame is widely used. However, in this technique, since the parameter is calculated for each frame alone, the correction parameter value fluctuates in a short time, and the corresponding correction parameter also fluctuates. Therefore, there has been a problem that the image after correction may flicker. In order to solve this problem, a method of filtering the correction parameters is used.

  However, when scenes are switched at a certain time of a moving image, such as an edited moving image, filtering is essentially unnecessary because there is no correlation between the preceding and succeeding scenes. Although a technique of not performing filtering when the parameter fluctuates more than a certain value may be used, there is a problem in performance because the parameter value does not always fluctuate greatly at scene switching.

In order to solve the above problem, a technique for determining a correction parameter from a degree of correlation between histograms of a current frame and a frame immediately before the current frame is disclosed (for example, Patent Document 1). Specifically, when the correlation degree of the histogram between the current frame and the immediately preceding frame is equal to or greater than a certain value, the correction parameter used for the immediately preceding frame is used. The correction parameter is determined from the histogram.
JP 2007-241224 A

  However, in the above-described technique, it is determined whether the scene is switched in consideration of only the correlation between the current frame and the immediately preceding frame, and the correction parameter used for the immediately preceding frame is applied as it is, or a new current frame is newly determined. It is determined whether or not a correction parameter adapted to is applied. For this reason, there is a problem that it is not possible to cope with the case where it is difficult to calculate an appropriate correction parameter from only the histogram.

  The present invention has been made in view of such circumstances, and is a moving image processing apparatus, a moving image processing method, and a program thereof that can be applied even when it is difficult to calculate an appropriate correction parameter from only a histogram. An object of the present invention is to provide a recording medium and a moving image display device.

  The moving image processing apparatus of the present invention, for a plurality of scenes, a recording unit that records the main histogram of the scene and a correction parameter suitable for the scene, a histogram calculation unit that calculates a histogram from the pixel value of the frame, Correlation degree calculating means for calculating the degree of correlation between the histogram calculated by the histogram calculating means and the histograms of a plurality of scenes recorded by the recording means, and calculating correction parameters based on the correlation degree calculated by the correlation degree calculating means Correction parameter calculation means for performing correction, and correction means for performing correction using the correction parameter calculated by the correction parameter calculation means.

  The moving image processing method of the present invention includes a recording step of recording a plurality of scenes in association with a main histogram of the scene and a correction parameter suitable for the scene, a histogram calculation step of calculating a histogram from the pixel value of the frame, A correlation calculation step for calculating the correlation between the histogram calculated by the histogram calculation step and the histograms of a plurality of scenes recorded by the recording step, and calculating a correction parameter based on the correlation calculated by the correlation calculation step And a correction step for performing correction using the correction parameter calculated in the correction parameter calculation step.

  The program according to the present invention includes a process of recording a plurality of scenes in association with main histograms of scenes and correction parameters suitable for the scenes, a process of calculating histograms from pixel values of frames, and a process of calculating histograms. A process for calculating a correlation between the calculated histogram and a histogram of a plurality of scenes recorded by the recording process, a process for calculating a correction parameter based on the correlation calculated by the process for calculating the correlation, And causing the computer to execute a process of performing correction using the correction parameter calculated by the process of calculating the correction parameter.

  The recording medium of the present invention is a computer-readable recording medium on which the program is recorded.

  A moving image display apparatus according to the present invention includes the moving image processing apparatus.

  According to the present invention, even when it is difficult to calculate an appropriate correction parameter from only a histogram, it is possible to perform appropriate correction on a moving image.

  Hereinafter, examples of embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic configuration example of a moving image processing apparatus according to the present embodiment. As illustrated, the moving image processing apparatus 10 according to the present embodiment includes a histogram calculation unit 101, a correlation degree calculation unit 102, a correction parameter calculation unit 103, a filter processing unit 104, and a correction processing unit 105.

  The histogram calculation unit 101 creates a frequency distribution from each pixel value. The number of floors may be from several floors to over a dozen floors. A digitized image is a collection of pixels arranged in the horizontal direction and the vertical direction, and a histogram is calculated by counting the number of pixels having the same rank among the pixel data.

  The correlation degree calculation unit 102 calculates the correlation degree between the two histograms. There are various methods for calculating the degree of correlation. For example, a method for calculating the degree of correlation using Kendall's rank correlation coefficient represented by the following equation (1) can be applied. In addition, as an application of the above method, a method that considers a match when the difference is equal to or smaller than a certain value can be applied. Note that the method for calculating the degree of correlation is not limited to this.

t = 4P / n / (n-1)-1 ... Formula (1)
P indicates the number of matching ranks.

  The correction parameter calculation unit 103 calculates a correction parameter based on the histogram. As a method for calculating the correction parameter, for example, the minimum value and the maximum value are estimated from an appropriate quantile of the histogram (for example, the 1/8 quantile and the 7/8 quantile, etc.), and the brightness correction parameter is set. And a method for obtaining a gamma correction parameter from the distribution of the kurtosis and quantile of the histogram. The correction parameter calculation method is not limited to this.

  The filter processing unit 104 performs low-pass filter processing (LPF) on the time axis so that the correction parameter does not fluctuate finely. Thereby, noise generated in the process of decoding the encoded image data can be reduced. The simplest implementation is a method of obtaining an average value for the past N. In addition, methods such as FIR (Finite Impulse Response) and IIR (Infinite Impulse Response) can be applied. The filter processing unit 104 performs a reset operation based on the magnitude relationship between the correlation degree and the threshold value. In the reset operation, past accumulated parameters are discarded.

  The correction processing unit 105 performs input image correction processing. There are various types of image processing, but image processing can be performed by a well-known method, and description thereof is omitted here.

(Embodiment 1)
FIG. 2 shows an example of the flow of moving image processing according to this embodiment. In the present embodiment, an example of the flow of moving image processing will be described with reference to the diagram shown in FIG.

  First, as pre-processing, for example, processing such as representative point extraction (or reduction) processing to reduce the scale of signal processing and color value luminance processing is performed to remove noise from the input signal. In general, the brightness of the color value is processed by the average value or the total value of the RGB components. The pre-processing is not limited to this. In the present embodiment, the pre-processing can be omitted.

  Next, the histogram calculation unit 101 obtains a histogram of the frame. The correction parameter calculation unit 103 calculates a correction parameter suitable for the frame based on the obtained histogram. At the same time, the correlation degree calculation unit 102 calculates the correlation degree of the histogram between the frame that is currently processed and the frame that was processed before that. If the correlation calculated by the correlation calculation unit 102 is less than or equal to a certain level, the filter is reset on the assumption that the scene has been switched. And the correction process part 105 performs a correction process using the correction parameter calculated | required from the above process.

  According to the present embodiment, since the scene change appears as a histogram change, it is possible to detect the scene change with high accuracy and low cost by obtaining the degree of correlation between the histograms of the preceding and succeeding frames. It is also possible to reset and initialize the filter appropriately according to the scene change.

(Embodiment 2)
FIG. 3 shows a schematic configuration example of the moving image processing apparatus according to the present embodiment. The difference from the first embodiment described above is that a recording unit 106 is provided. The recording unit 106 stores a representative histogram of each scene and a correction parameter suitable for the scene as a parameter template in association with a plurality of scenes selected in advance. As the scene, for example, a representative subject or photographing environment such as a landscape, a portrait, a night view, or a sports landscape may be selected. In addition, the same code | symbol is attached | subjected to the function similar to the function mentioned above, and description is abbreviate | omitted.

  FIG. 4 shows an example of the flow of moving image processing according to this embodiment. In the present embodiment, an example of the flow of moving image processing will be described with reference to the diagram shown in FIG.

  First, as pre-processing, for example, processing such as representative point extraction (or reduction) processing to reduce the scale of signal processing and color value luminance processing is performed to remove noise from the input signal. In general, the brightness of the color value is processed by the average value or the total value of the RGB components. The pre-processing is not limited to this. In the present embodiment, the pre-processing can be omitted.

  Then, the histogram calculation unit 101 obtains a histogram of the frame, and the correlation degree calculation unit 102 calculates the degree of correlation between the histogram of the frame and the histogram of each scene stored in the parameter template. A method similar to that in the first embodiment can be applied to the correlation degree calculation method.

  An appropriate parameter is obtained from the degree of correlation calculated above. As methods for obtaining appropriate parameters, the following two methods can be given as examples. Note that the method for obtaining the parameters for correction is not limited to the following method.

  As a first method for obtaining a parameter, there is a method of adopting a parameter of a scene having the highest correlation among scenes recorded in advance in a parameter template. In this method, the degree of correlation between the histogram obtained from the frame and the histogram of each scene registered in advance is obtained, and the correction parameter stored corresponding to the scene with the highest degree of correlation is used to correct the frame. To do.

  As a second method for obtaining parameters, there is a method for obtaining a weighted average of each parameter according to the degree of correlation. In this method, the degree of correlation between the histogram obtained from the frame and the histogram of each scene registered in advance is obtained. Then, a weighted average of parameters stored corresponding to each scene is obtained according to the obtained degree of correlation with each scene, and is used as a new correction parameter.

  Next, the correction processing unit 105 performs correction processing using the correction parameters obtained from the above processing.

  In this embodiment, the correction parameter corresponding to the scene is not obtained from the image, but only the scene is detected from the image, and the parameter created or stored in advance is used. For this reason, even if it is difficult to calculate an appropriate correction parameter from only the histogram, it is possible to perform an appropriate correction process.

  In the above-described embodiment, an example in which a scene of each frame is detected and a correction parameter suitable for each frame is obtained has been described. However, the present invention is not limited to this. For example, as in the first embodiment described above, scene change is detected in advance by detecting the degree of correlation between the preceding and succeeding frames. Only for the first frame of each scene, it is determined which scene is recorded in the parameter template. The parameters recorded corresponding to the scene can be applied to all frames of the scene. Here, as a method for obtaining the parameters, it is also possible to apply the method for obtaining from the weighted average as described above.

  Also, for example, as in the first embodiment described above, scene change is detected by detecting the degree of correlation between the previous and subsequent frames. For each scene, the histogram of all frames is averaged to determine which scene recorded in the parameter template is applicable. It is also possible to apply the parameters recorded corresponding to the scene to all frames of the scene. Here, as a method for obtaining the parameters, it is also possible to apply the method for obtaining from the weighted average as described above.

(Embodiment 3)
FIG. 5 shows a schematic configuration example of the moving image processing apparatus according to the present embodiment. The difference from the first embodiment described above is that a correction parameter recording unit 107 is provided. The correction parameter recording unit 107 records the parameters applied to the processed frame before the currently processed frame. In addition, the same code | symbol is attached | subjected to the function similar to the function mentioned above, and description is abbreviate | omitted.

  FIG. 6 shows an example of the flow of moving image processing according to this embodiment. In the present embodiment, an example of the flow of moving image processing will be described with reference to the diagram shown in FIG.

  First, as pre-processing, for example, processing such as representative point extraction (or reduction) processing to reduce the scale of signal processing and color value luminance processing is performed to remove noise from the input signal. In general, the brightness of the color value is processed by the average value or the total value of the RGB components. The pre-processing is not limited to this. In the present embodiment, the pre-processing can be omitted.

  Next, the histogram calculation unit 101 obtains a histogram of the frame. The correlation level calculation unit 102 calculates the correlation level of a histogram between a frame that is currently being processed and a frame that was previously processed. When the degree of correlation calculated by the degree-of-correlation calculation unit 102 is equal to or greater than a certain level, the parameter used immediately before recorded in the correction parameter recording unit 107 is employed to perform correction processing.

  On the other hand, when the degree of correlation calculated by the degree-of-correlation calculation unit 102 is below a certain level, it is assumed that the scene is switched between the currently processed frame and the previously processed frame, as described in the second embodiment. The correction parameter is obtained in the same manner as in. That is, the degree of correlation between the histogram of the frame obtained by the histogram calculation unit 101 and the histogram of each scene stored in the parameter template is calculated, and an appropriate parameter is obtained from the calculated degree of correlation. The parameters recorded in the correction parameter recording unit 107 are updated with the new parameters obtained in this way. Then, correction processing is performed using the updated parameters.

  According to the present embodiment, parameters are updated only when a scene change occurs, so that some processing such as correlation calculation is performed only when necessary. Therefore, by reducing processing time and current required for processing, the cost can be reduced. Can be reduced.

  The moving image display apparatus according to the present invention includes any of the moving image processing apparatuses described above. The moving image display device has a general function for operating as a moving image display device such as a display unit that displays a moving image.

  Note that the program for the CPU to execute the processing shown in each figure constitutes a program according to the present invention. As a computer-readable recording medium for recording the program, a semiconductor storage unit, an optical and / or magnetic storage unit, or the like can be used. By using such a program and recording medium in a system having a configuration different from that of each of the above-described embodiments and causing the CPU to execute the program, substantially the same effects as those of the present invention can be obtained.

  Although specifically described based on the preferred embodiments, the present invention is not limited to the above-described moving image processing apparatus, moving image processing method, program recording medium, and moving image display apparatus. It goes without saying that various changes can be made without departing from the scope.

It is a block diagram which shows the schematic structural example of the moving image processing apparatus which concerns on this embodiment. It is a figure which shows the example of the flow of the moving image process which concerns on this embodiment. It is a block diagram which shows the schematic structural example of the moving image processing apparatus which concerns on this embodiment. It is a figure which shows the example of the flow of the moving image process which concerns on this embodiment. It is a block diagram which shows the schematic structural example of the moving image processing apparatus which concerns on this embodiment. It is a figure which shows the example of the flow of the moving image process which concerns on this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Moving image processing apparatus 101 Histogram calculation part 102 Correlation degree calculation part 103 Correction parameter calculation part 104 Filter processing part 105 Correction processing part 106 Recording part 107 Correction parameter recording part

Claims (13)

For a plurality of scenes, recording means for recording the main histogram of the scene and correction parameters suitable for the scene,
A histogram calculating means for calculating a histogram from the pixel values of the frame;
Correlation degree calculating means for calculating the degree of correlation between the histogram calculated by the histogram calculating means and the histograms of a plurality of scenes recorded by the recording means;
Correction parameter calculating means for calculating a correction parameter based on the correlation degree calculated by the correlation degree calculating means;
And a correction unit that performs correction using the correction parameter calculated by the correction parameter calculation unit. A filter processing means for performing filter processing;
The correlation calculation means calculates the correlation between the histogram of the current frame and the histogram of the past frame,
2. The moving image according to claim 1, wherein the filter processing means initializes the filter when the degree of correlation between the current frame and the past frame calculated by the correlation degree calculating means is below a certain level. Processing equipment. A correction parameter recording means for recording past correction parameters;
The correlation calculation means calculates the correlation between the histogram of the current frame and the histogram of the past frame,
The correction parameter calculation means calculates a correction parameter when the degree of correlation between the current frame and the past frame calculated by the correlation degree calculation means is below a certain level,
The correction parameter recording means calculates the recorded correction parameter by the correction parameter calculation means when the degree of correlation between the current frame and the past frame calculated by the correlation degree calculation means is below a certain level. Update to the correction parameter
The moving image processing apparatus according to claim 1, wherein the correction unit performs correction using the correction parameter recorded by the correction parameter recording unit.   The correction parameter calculation means uses the correction parameter recorded corresponding to the scene with the highest correlation calculated by the correlation calculation means as the correction parameter of the frame. The moving image processing apparatus according to any one of the above.   The said correction parameter calculation means calculates a correction parameter by a weighted average based on the correlation degree of the said scene calculated by the said correlation degree calculation means, The any one of Claim 1 to 3 characterized by the above-mentioned. Moving image processing apparatus. For a plurality of scenes, a recording step for recording a main histogram of the scene and a correction parameter suitable for the scene,
A histogram calculation step for calculating a histogram from the pixel values of the frame;
A correlation calculation step for calculating a correlation between the histogram calculated by the histogram calculation step and the histograms of a plurality of scenes recorded by the recording step;
A correction parameter calculating step for calculating a correction parameter based on the correlation degree calculated by the correlation degree calculating step;
And a correction step of performing correction using the correction parameter calculated in the correction parameter calculation step. A filtering step for performing filtering,
The correlation calculation step calculates a correlation between a histogram of a current frame and a histogram of a past frame,
7. The moving image according to claim 6, wherein the filter processing step initializes the filter when the correlation degree between the current frame and the past frame calculated by the correlation degree calculation step is below a certain level. Processing method. A correction parameter recording step of recording past correction parameters;
The correlation calculation step calculates a correlation between a histogram of a current frame and a histogram of a past frame,
The correction parameter calculating step calculates a correction parameter when the degree of correlation between the current frame and the past frame calculated by the correlation degree calculating step is less than or equal to a certain level,
The correction parameter recording step calculates the recorded correction parameter by the correction parameter calculation step when the degree of correlation between the current frame and the past frame calculated by the correlation degree calculation step is below a certain level. Update to the correction parameter
8. The moving image processing method according to claim 6, wherein the correction step performs correction using the correction parameter recorded in the correction parameter recording step.   9. The correction parameter calculating step uses the correction parameter recorded corresponding to the scene having the highest correlation calculated in the correlation calculation step as the correction parameter of the frame. The moving image processing method according to any one of the above.   9. The correction parameter calculation step according to claim 6, wherein the correction parameter calculation step calculates a correction parameter by a weighted average based on the correlation degree of the scene calculated by the correlation degree calculation step. Video processing method. For a plurality of scenes, a process of recording the main histogram of the scene and the correction parameters suitable for the scene,
A process of calculating a histogram from the pixel values of the frame;
A process for calculating the degree of correlation between the histogram calculated by the process of calculating the histogram and the histograms of a plurality of scenes recorded by the process of recording;
A process of calculating a correction parameter based on the correlation degree calculated by the process of calculating the correlation degree;
The program which makes a computer perform the process which correct | amends using the correction parameter calculated by the process which calculates the said correction parameter.   The computer-readable recording medium which recorded the program of Claim 11.   A moving image display apparatus comprising the moving image processing apparatus according to claim 1.

Images