JP4265986B2 - Image processing method, apparatus, and program - Google Patents

Description

  The present invention relates to an image processing method and apparatus for correcting the image quality of a moving image, and a program therefor.

  It is widely performed to correct the image quality of the original image so that the digital image looks more beautiful. Examples of these image quality corrections include white balance correction, exposure correction, saturation correction, contrast correction, sharpness correction, and color correction. As a method for correcting a still image, it is known to extract a feature amount from a still image, obtain a correction value based on the extracted feature amount, and correct the original image using the correction value. The feature amount extracted from the still image includes, for example, the average luminance value of the original image, the average luminance value in a dark area in the original image, and the main subject ( Average luminance value of a person's face, etc., an average saturation value in a high saturation area in the original image, an RGB average gradation value in a bright area in the original image, a hue histogram in the original image, and the like.

  The above-described still image quality correction method can be applied to moving image quality correction. Specifically, each frame constituting the moving image (hereinafter referred to as a frame image) is regarded as a still image, a feature amount is extracted from each frame image, and a correction value of each frame image is obtained based on this feature amount, It is conceivable to correct the moving image by correcting the frame image.

  However, the frame image of the moving image changes in the time direction, and its feature amount also fluctuates. Therefore, when the still image correction method is applied to each frame image of the moving image, it is composed of the corrected frame image. There is a problem that the moving image (corrected moving image) flickers. In order to solve this problem, Patent Document 1 proposes a method of smoothing correction value transition of a frame image of a moving image. Specifically, with respect to the correction value obtained from the frame image of the moving image, the difference between the correction value of the current frame image and the correction value of the frame image before the current frame image is limited within a predetermined threshold. There has been proposed a method for suppressing flicker occurring in a corrected moving image by obtaining a target correction value. There is also a method in which a value obtained by taking the average (moving average) of the correction value of the current frame image and the correction value of the frame image before and / or after the current frame image is used as the target correction value of the current frame image. Conceivable.

On the other hand, there are a plurality of different scenes, such as an indoor scene and an outdoor scene, and the frame images of these different scenes are not related to each other. Therefore, if the transition of the correction value of the frame image is suppressed in order to eliminate the flicker of the corrected moving image, the purpose of the frame image that becomes the cut points (first and last frame images of one scene) indicating the switching of the scenes There is a problem that the correction value is affected by the correction value of the frame image of a scene different from the scene of the frame image. For example, the last frame image of scene 1 is the first frame image of the scene immediately after scene 1 (scene 2), although the frame images adjacent in the time direction are not related to each other. First, when the target correction value is obtained by taking a moving average of the correction values of the frame image, for example, to eliminate the flicker of the corrected moving image, the target correction value of the last frame image of the scene 1 becomes the frame image and The average value of the correction values obtained from the frame images before and after that (including the first frame image of the scene 2) is obtained, and the target correction value of the last frame image of the scene 1 is not appropriate. In order to solve this problem, Patent Document 1 discloses a method for detecting a cut point indicating a scene change in a moving image and smoothing a transition of a correction value of a frame image to obtain a target correction value of the frame image. A method has been proposed in which a correction value of a frame image of a scene different from the frame image of interest is not used.
JP 2002-262303 A

  However, in a moving image, the frame image may change suddenly in the same scene. For example, as shown in FIG. 13, three frame images A, B, and C obtained by imaging a person who moves at high speed up are frame images of the same scene, but the contents of the images are quite different. . The method proposed in Patent Document 1 can determine an appropriate target correction value for a frame image to be a cut point, but when determining a target correction value for a frame image such as the frame image B shown in FIG. If it is determined that the frame image B is a cut point and the correction value obtained from the frame image B is set as the target correction value of the frame image B, the target correction value between the frame image B and the frame images A and C Is not smoothed, flickering occurs in the same scene of the corrected moving image. On the other hand, if it is determined that the frame image B is not a cut point and the average value of the correction values of the frame images A, B, and C is set as the target correction value of the frame image B, the frame image B and the frame image A, The difference between C is not reflected and the image quality is not good.

  In view of the above circumstances, the present invention eliminates the flicker of a corrected moving image and is not limited to a frame image serving as a cut point indicating switching of scenes. It is an object of the present invention to provide an image processing method and apparatus capable of performing appropriate correction, and a program therefor.

The first image processing method of the present invention obtains a temporary correction value for correcting the image quality of each frame image of a plurality of frame images constituting a moving image,
In the image processing method for obtaining a value obtained by adjusting each temporary correction value so that each temporary correction value changes smoothly in the time direction as a target correction value of the frame image,
The frame image is determined for each frame image as a cut point indicating the probability of being a cut point indicating a scene change in the moving image,
The target correction value is obtained by adjusting the degree of adjustment of the frame image with respect to the temporary correction value according to the size of the cut point likelihood of the frame image.

  Here, the temporary correction value of the frame image means a correction value for correcting the image quality of the frame image obtained from the frame image of interest.

  Further, adjusting the temporary correction value so that the temporary correction value is smoothly converted in the time direction means suppressing the transition of the temporary correction value, specifically, the temporary correction of the frame image of interest. The value obtained by averaging the value and the provisional correction value of the frame image before and / or after this frame image is used as the target correction value of the frame image to be noticed, or the provisional values of a plurality of consecutive frame images A value obtained by applying a low-pass filter to the correction value as a target correction value for each of the plurality of frame images, or a value obtained by performing interpolation processing on the temporary correction values for a plurality of consecutive frame images Can be used as the respective target correction values of the plurality of frame images. Note that the process of applying the low-pass filter, the interpolation process, etc. are for smoothing the change of the temporary correction value, so use a high-order low-pass filter as the filter and the spline interpolation method as the interpolation method. Is preferred.

  Various adjustments of the temporary correction value can be considered as described above. As a result, the difference between the temporary correction value of the target frame image and the temporary correction value of the previous and / or subsequent frame image is the result. Therefore, the temporary correction value of one frame image is affected by the temporary correction value of the previous and / or subsequent frame image. Therefore, the degree to which the temporary correction value of the frame image is adjusted means that the target correction value of the frame image of interest is the same as the degree to which the temporary correction value of the previous and / or subsequent frame image is affected. That is, according to the first image processing method of the present invention, when the target correction value of the frame image is obtained, the preliminary correction of the frame image before and / or after the frame image is performed according to the likelihood of the cut point of the frame image. It adjusts the degree of the influence of the value.

  The “cut-likeness” indicates the size of the probability that the frame image of interest is a cut point indicating a scene change in the moving image. The cut point is the last point of one scene in the moving image. The cut image of the frame image having a large image change in the same scene as shown in FIG. 13, whereas the cut point likelihood of the frame image of the cut point has the maximum value. The pointiness often becomes smaller than this maximum value.

  The cut point likelihood relates to the magnitude and / or mode of the feature amount of the frame image in the time direction, and the cut point likelihood of each frame image is obtained based on the variation of the feature amount of the frame image. it can. For example, when the difference between the feature amount of the frame image of interest and the feature amount of the frame image before or immediately after it is equal to or greater than a predetermined first threshold, the likelihood of the cut point of the frame image of interest is set to 1 as the maximum, When the difference between the feature amount of the frame image of interest and the feature amount of the frame image immediately before or immediately after it is equal to or smaller than a predetermined second threshold (second threshold <first threshold), When the cut point likelihood is set to the minimum 0 and the difference between the feature value of the frame image to be noticed and the feature value of the frame image immediately before or immediately after is between the first threshold value and the second threshold value, attention is paid. It can be determined that the cut point likelihood of the frame image is between 0 and 1. Further, the present invention is not limited to the difference between the feature amounts of the frame images. For example, the difference between the feature amount of the frame image of interest and the average value of the feature amounts of the plurality of frame images before and / or after that. The cut point likelihood may be obtained with reference to.

  In addition, the feature amount used when calculating the likelihood of a cut point may be an average pixel value in an entire frame image or an area excluding a high pixel value area and a low pixel value area. It is preferable to use the shape of a histogram of values. This is because the shape of the histogram of the pixel value of the frame image as the cut point is greatly different from the shape of the histogram of the pixel value of the previous or subsequent frame image, for example, a frame in a scene as shown in FIG. In order to change the shape of the histogram of the pixel values of the image according to the proportion of the subject in the entire image and not to change as much as the cut point, the variation of the shape of the histogram of the pixel values of the frame image is directly This is because it is possible to reflect the likelihood of an image cut point. Further, as the difference in the shape of the histogram, the correlation between the shapes of the histogram can be used so that the larger the correlation is, the smaller the difference is.

  In addition, since the above description regarding the likelihood of a cut point and the likelihood of a cut point can be applied to all image processing methods and apparatuses and programs of the present invention, in the following description, the likelihood of a cut point and the likelihood of a cut point The detailed description is omitted for obtaining.

  In the first image processing method of the present invention, when obtaining the target correction value, the degree of adjustment is weakened in a frame image having a higher likelihood of the cut point, that is, before and / or after the frame image. It is preferable to adjust so as to weaken the influence of the frame image.

In the second image processing method of the present invention, the feature amount of the frame image is calculated for each frame image of the plurality of frame images constituting the moving image,
Target feature value calculation processing for obtaining a target value of the feature value of the frame image so that the feature value of each corrected frame image obtained by correcting each frame image changes smoothly with a predetermined smoothness in the time direction. For each frame image,
In the image processing method for obtaining a target correction value for each frame image so that the feature amount of the frame image is substantially equal to the target feature amount of the frame image,
The frame image is determined for each frame image as a cut point indicating the probability of being a cut point indicating a scene change in the moving image,
The target feature amount is obtained by adjusting the degree of smoothness when obtaining the target feature amount of the frame image according to the size of the cut point likelihood of the frame image.

  Here, it is preferable to adjust the frame image having a greater likelihood of the cut point so that the smoothness is weakened.

  As described above, when correcting the image quality of a moving image, each frame image constituting the moving image is regarded as a still image, and the feature amount is extracted for each frame image, and obtained based on the extracted feature amount. There is a problem that the corrected moving image flickers only by correcting the frame image with the corrected value. In the second image processing method of the present invention, in order to solve this problem, the target feature amount of the frame image is set so that the feature amount of each corrected frame image changes smoothly with a predetermined smoothness in the time direction. A target feature amount calculation process for obtaining a value is obtained for each frame image, and a target correction value for each frame image is obtained so that the feature amount of the frame image becomes the target feature amount of the frame image. Further, the degree of smoothness at the time of obtaining the target feature amount of the frame image is adjusted according to the size of the cut point likelihood of the frame image of interest.

Here, the adjustment method of the smoothness level differs depending on the method of calculating the target feature amount. For example, in the second image processing method of the present invention, the target feature amount calculation processing obtains a temporary correction value for correcting the image quality of the frame image from the frame images,
Calculating a feature amount of the temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
The frame corresponding to the temporarily corrected frame image is a value obtained by adjusting the feature value of each temporarily corrected frame image so that the feature value of each temporarily corrected frame image changes smoothly in the time direction. The processing can be obtained as the target feature amount of the image. In this case, the frame before and / or after the frame image is adjusted by adjusting the degree of the adjustment performed when obtaining the target feature amount of the frame image according to the size of the cut point likelihood of the frame image. The degree of influence of the image may be adjusted. Desirably, the adjustment is performed so that the degree of adjustment with respect to the feature amount of the temporarily corrected frame image is weakened in the frame image having a larger cut point likelihood. In this case, the degree of adjustment corresponds to the degree of smoothness.

  Here, adjusting the feature amount of the temporarily corrected frame image so that the feature amount of the temporarily corrected frame image changes smoothly in the time direction means that the feature amount transition of the corrected frame image is suppressed. Specifically, for example, the feature amount of the temporarily corrected frame image of interest and the feature amount of the temporarily corrected frame image before and / or after the temporarily corrected frame image are averaged (weighted average is used). The target feature value of the temporarily corrected frame image to be noticed, or a value obtained by applying a low-pass filter to the feature values of a plurality of consecutive temporarily corrected frame images. Each of the temporarily corrected frame images has a target feature value, or a value obtained by performing an interpolation process on the feature values of a plurality of consecutive temporarily corrected frame images is a plurality of temporarily corrected frame images. It is possible to use a method such as or a target characteristic quantity of each of the over-time image. Note that the process of applying the low-pass filter, the interpolation process, and the like are for smoothing the change in the feature amount of the temporarily corrected frame image. Therefore, the filter is a high-order low-pass filter, and the interpolation method is a spline. It is preferable to use an interpolation method. Therefore, for example, a temporarily corrected frame image that pays attention to a value obtained by taking a weighted average of the feature amount of the temporarily corrected frame image to be noticed and the feature amount of the preceding and / or subsequent temporarily corrected frame image is obtained. When the target feature amount of the corresponding frame image is used, the weight of the feature amount of the temporarily corrected frame image to be noticed is increased as the cut point likelihood of the frame image increases (that is, the previous and / or subsequent temporarily corrected frame image). (Weighting of the feature amount) may be reduced) and weighted averaging may be performed. The smoothing degree in this case corresponds to the weight of the feature amount of the pre-corrected frame image before and / or after.

  Further, for example, a representative frame image (not necessarily a focused frame image) is selected from the frame image of interest and the frame images before and / or after the frame image, and the temporarily corrected frame image corresponding to the representative frame image is selected. When smoothing is performed by weighted averaging the feature amount of the frame image and the feature amount of the temporarily corrected frame image of the frame image of interest, the larger the cut point likelihood of the frame image is, the temporarily corrected of the frame image of interest is A weighted average may be performed by increasing the weight of the feature amount of the frame image (that is, decreasing the weight of the feature amount of the temporarily corrected frame image of the representative frame image). The smoothing degree in this case corresponds to the feature amount weight of the temporarily corrected frame image of the representative frame image. In this case, it is preferable to select the brightest frame image or the frame image with the smallest temporary correction value as the representative frame image so as to improve the image quality.

  As described above, various methods can be used to adjust the target feature amount. As a result, the difference between the target feature amount of the target frame image and the target custom order amount of the previous and / or subsequent frame image is the result. Therefore, the target feature amount of one frame image is affected by the feature amount of the temporarily corrected frame image corresponding to the previous and / or subsequent frame image. Therefore, the degree of adjustment of the feature amount of the temporarily corrected frame image is determined so that the target feature amount of the frame image of interest is the previous and / or subsequent frame image (the feature amount of the previously corrected representative frame image of the representative frame image). When the target feature amount is obtained using the above, it means the same as the degree affected by the feature amount of the temporarily corrected frame image corresponding to the representative frame image).

In the second image processing method of the present invention, the target feature amount calculation processing obtains a temporary correction value for correcting the image quality of the frame image from each frame image,
Calculating a feature amount of the temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
The frame corresponding to the temporarily corrected frame image is a value obtained by adjusting the feature value of each temporarily corrected frame image so that the feature value of each temporarily corrected frame image changes smoothly in the time direction. As a temporary target feature of the image,
A process of obtaining a value obtained by weighted addition of the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image as the target feature amount of the frame image can be performed. In this case, by adjusting the weight of the temporary target feature amount used when obtaining the target feature amount of the frame image according to the size of the cut point likelihood of the frame image, Alternatively, the degree of influence of the subsequent frame image may be adjusted, and desirably, the frame image having a larger cut point likelihood is adjusted so that the weight of the temporary target feature amount is reduced.

  Here, “adjusting the weight of the provisional target feature amount” means adjusting the weight of the provisional target feature amount when the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image are weighted and added. Therefore, if the weight of the temporary target feature amount is adjusted, the weight of the feature amount is inevitably adjusted. Conversely, adjusting the weight of the feature amount also adjusts the weight of the temporary target feature amount. Note that the weight of the temporary target feature amount in this case corresponds to the smoothness level in the present invention.

  In the second image processing method of the present invention, a target feature amount is obtained for each frame image constituting the moving image, and the target correction value of each frame image is set so that the feature amount of the frame image becomes the target feature amount. As described above, the processing for obtaining the target feature amount may be performed on all the frame images by the same method. However, in order to save the processing time, each frame image constituting the moving image is used. A target feature amount may be obtained only for the selected reference frame image, and the target feature amount of each frame image may be obtained by interpolating the obtained target feature amount of the reference frame image.

  In the image processing method of the present invention, it is preferable to perform processing until obtaining the target correction value using a reduced image of a frame image constituting the moving image.

  The image processing method of the present invention may perform processing up to correcting the frame image corresponding to the target correction value using the target correction value.

A first image processing apparatus according to the present invention includes a temporary correction value calculating unit that calculates a temporary correction value for correcting the image quality of a frame image for each frame image of a plurality of frame images constituting a moving image;
In an image processing apparatus comprising: a target correction value calculating unit that obtains a value obtained by adjusting each temporary correction value so as to change smoothly in the time direction as a target correction value of the frame image.
The frame image further comprises cut point likelihood calculating means for determining, for each of the frame images, the cut point likelihood indicating the probability that the frame image is a cut point indicating a scene change in the moving image,
The target correction value calculation means finds the target correction value by adjusting the degree of adjustment of the frame image with respect to the temporary correction value according to the size of the cut point likelihood of the frame image. To do.

  It is preferable that the target correction value calculation unit adjusts the degree of adjustment to be weaker for a frame image having a larger cut point likelihood.

The second image processing apparatus of the present invention includes a feature amount calculating unit that calculates a feature amount of each frame image of a plurality of frame images constituting the moving image,
The target value of the feature value of the frame image is obtained so that the feature value of each corrected frame image obtained by correcting each of the frame images changes smoothly with a predetermined smoothness in the time direction. Target feature amount calculating means for
In an image processing apparatus comprising target correction value calculation means for calculating a target correction value for each frame image so that the feature amount of the frame image is substantially equal to the target feature amount of the frame image.
The frame image is provided with a cut point likelihood calculating means for determining, for each of the frame images, a cut point likelihood indicating a probability of being a cut point indicating a scene change in the moving image,
The target feature amount calculating unit adjusts the smoothness degree when the target feature amount of the frame image is calculated according to the size of the cut point likelihood calculated by the cut point likelihood calculating unit. It is characterized in that the quantity is obtained.

  Here, it is preferable that the target feature amount calculating means adjusts so that the degree of smoothness is weakened in a frame image having a higher likelihood of a cut point.

In the second image processing apparatus of the present invention, the target feature amount calculation means includes provisional correction value calculation means for obtaining a provisional correction value for correcting the image quality of the frame image from each frame image;
Temporary correction means for calculating a feature amount of a temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
The frame corresponding to the temporarily corrected frame image is a value obtained by adjusting the feature value of each temporarily corrected frame image so that the feature value of each temporarily corrected frame image changes smoothly in the time direction. A feature amount smoothing means for obtaining the target feature amount of the image,
It is preferable that the feature amount smoothing unit adjusts the degree of adjustment according to the size of the cut point likelihood.

In the second image processing apparatus of the present invention, the target feature amount calculation means includes provisional correction value calculation means for obtaining a provisional correction value for correcting the image quality of the frame image from each frame image;
Temporary correction means for calculating a feature amount of a temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
The frame corresponding to the temporarily corrected frame image is a value obtained by adjusting the feature value of each temporarily corrected frame image so that the feature value of each temporarily corrected frame image changes smoothly in the time direction. Feature amount smoothing means for obtaining a temporary target feature amount of an image;
A weighted addition means for obtaining a value obtained by weighted addition of the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image as the target feature amount of the frame image;
It is preferable that the weighted addition means adjusts the weight of the temporary target feature amount according to the size of the cut point likelihood.

In the second image processing apparatus of the present invention, the target feature amount calculating means performs a process of obtaining the target feature amount only for a reference frame image selected from each frame image constituting the moving image. What to do,
Interpolation means for obtaining the target feature amount of each frame image constituting the moving image by performing an interpolation process on each target feature amount obtained using the reference frame image. Is preferred.

  In the second image processing apparatus of the present invention, it is preferable that each of the means performs each process using a reduced image of a frame image constituting the moving image.

  The first and second image processing apparatuses of the present invention may further include correction means for correcting the frame image corresponding to the target correction value using the target correction value.

  Note that the image processing method of the present invention may be provided as a program to be executed by a computer.

  According to the first image processing method and apparatus of the present invention, provisional correction values are obtained for a plurality of frame images constituting a moving image, and values obtained by adjusting changes in the provisional correction values in the time direction are obtained. When setting the target correction value, the degree of adjustment is adjusted according to the size of the cut point likelihood of each frame image. By doing so, it is possible to adjust the degree to which the target correction value of the frame image is influenced by the temporary correction value of the previous and / or subsequent frame image. For example, if a frame image having a larger cut point likelihood is adjusted so that the degree of adjustment is weakened, the target correction value of the frame image that is the cut point (the frame image having the largest cut point likelihood) is set to the previous and / or It can be prevented from being influenced by the provisional correction value of the subsequent frame image, and an appropriate correction value of the frame image at the cut point can be obtained. In addition, the target correction value of a frame image with a small change in the same scene (a frame image with a low likelihood of a cut point; hereinafter referred to as a normal frame image) is affected by the temporary correction values of the previous and / or subsequent frame images, and the time direction Therefore, the flicker occurring in the corrected moving image can be eliminated. Furthermore, as shown in FIG. 13, a frame image that is the same scene but has a large image change (a frame that has a cut point likelihood that is smaller than the cut point likelihood of the cut point frame image but that is larger than the normal frame image cut point likelihood). In the case of an image), the target correction value is influenced by the temporary correction value of the previous and / or subsequent frame image, while the degree of the influence is smaller than that of the normal frame image. Therefore, while correcting flicker in the corrected moving image, appropriate correction can be obtained even in a frame image that is in the same scene but has a large image change.

  The second image processing method and apparatus according to the present invention each frame image so that the feature amount of each corrected frame image obtained by correcting each frame image changes smoothly with a predetermined smoothness in the time direction. When the target value of the feature value of the frame image is obtained, the target feature amount is obtained by adjusting the smoothness according to the likelihood of the cut point of the frame image, and the feature amount of the frame image before correction is the target feature amount. The target correction value is obtained so that By doing so, it is possible to adjust the degree to which the target feature amount of the frame image is influenced by the feature amount of the previous and / or subsequent frame image. For example, it can be adjusted so that the influence of the frame image before and / or after the frame image is weaker as the target feature amount of the frame image having a higher cut point likelihood is the same effect as the first image processing method and apparatus. Can be obtained.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image processing apparatus A according to the first embodiment of the present invention. The configuration of the image processing apparatus A shown in FIG. 1 is realized by executing an image processing program read into the auxiliary storage device on a computer (for example, a personal computer). The image processing program is stored in an information storage medium such as a CD-ROM or distributed via a network such as the Internet and installed in a computer. The image processing apparatus A according to the present embodiment is connected to a DVD (Digital Versatile Disc player, a computer, a game machine, a DV (Digital Video) camera, or the like, and captures a moving image or distributes it via a network such as the Internet. The moving image is obtained by receiving the moving image, reading the moving image from a recording medium such as a DVD on which the moving image is recorded, etc. Further, as described above, it is performed on the digital image. There are various image quality corrections, and the image processing apparatus A of the present embodiment performs exposure correction on a moving image as an example.

  As shown in FIG. 1, an image processing apparatus A according to this embodiment includes an image input unit 1 that acquires a plurality of frame images constituting a moving image, and a reduction unit 10 that reduces a frame image to obtain a reduced frame image. A temporary correction value calculating means 20 for obtaining a temporary correction value from the reduced frame image, a target correction value calculating means 40 for obtaining a target correction value of the frame image based on the temporary correction value obtained by the temporary correction value calculating means 20, And a correction unit 50 that corrects each frame image using the target correction value obtained by the target correction value calculation unit 40 to obtain a corrected moving image.

  The temporary correction value calculation means 20 calculates a temporary correction value as follows. First, the temporary correction value calculation unit 20 creates a density histogram shown in FIG. 2 for the reduced frame image acquired by the reduction unit 10. At this time, α times the number of pixels of the reduced frame image is m, the m-th density value from the higher density Y is Ymax, and the m-th density value from the lower density Y is Ymin. ) To obtain the intermediate value Y0 of the histogram.

Y0 = (Ymin + Ymax) / 2 (1)

Next, the temporary correction value calculation unit 20 uses the calculated intermediate value Y0 to obtain a correction value γ0 for exposure correction of the reduced frame image according to the equation (2).

γ0 = Yd−Y0 (2)

Here, Yd is a target density, and may be determined by estimation from information on a human skin region under a standard light source, for example. Or you may make it obtain | require using the method described in gazettes, such as Unexamined-Japanese-Patent No. 2000-101860, Unexamined-Japanese-Patent No. 2000-196890, etc., for example.

  The temporary correction value calculating unit 20 outputs the correction value γ0 obtained for each frame image to the target correction value calculating unit 40 as a temporary correction value of each frame image.

  The target correction value calculation means 40 adjusts the temporary correction value γ0 of each frame image so that each temporary correction value γ0 calculated by the temporary correction value calculation means 20 changes smoothly in the time direction, and the target correction value γ1. Is what you get. Here, as an example, the target correction value calculating means 40 performs weighted addition of the temporary correction value γ0 of the frame image of interest and the average value of the temporary correction value γ0 of the frame image and one frame image before and after the frame image. Although the temporary correction value is adjusted, the frame image used for the adjustment may be a frame image only before or after the frame image of interest, or the number thereof may be increased or decreased. Also, the adjustment method of the temporary correction value is not limited to the addition, and a method such as applying a low-pass filter to the temporary correction value or performing spline interpolation may be used.

  As shown in FIG. 1, the target correction value calculation means 40 in the present embodiment is, for each frame image, the likelihood of a cut point that is the probability that the frame image is a cut point indicating a scene change. Cut point likelihood calculating means 25 for calculating β0, and an adjustment degree β (details will be described later) for adjusting the temporary correction value γ0 according to the cut point likelihood β0 calculated by the cut point likelihood calculating means 25. Using the adjustment degree calculation means 30, the temporary correction value γ0, and the adjustment degree β, the frame image of interest and the average value of the temporary correction value γ0 of the frame image of interest and one frame image before and after the frame image of interest are weighted. And adding means 35 for obtaining a target correction value γ1 of the frame image to be noted by addition.

  The cut point likelihood calculating means 25 in the present embodiment calculates the cut point likelihood of each frame image based on the shape of the histogram of density values of the frame image. Specifically, the cut point likelihood calculating unit 25 first obtains a histogram of density values for each reduced frame image of each frame image obtained by the reducing unit 10. FIG. 3A shows an example of a histogram of each frame image (here, a reduced frame image) calculated by the cut point likelihood calculating means 25 (in the figure, as an example, six consecutive frames: f1, The histograms of f2, f3, f4, f5 and f6 are shown). When a subject is captured by an imaging device such as a digital video camera or a digital still camera capable of shooting a moving image to obtain a moving image, a density step unrelated to the subject may occur between frame images due to the performance of the imaging device. For this reason, the cut point likelihood calculation unit 25 of the present embodiment calculates the representative value (here, the average value of the density values) of each frame image in order to eliminate the density step between the frame images due to the performance of the imaging apparatus. Further, calculation is performed and the histogram of each frame image is translated on the density value axis so that the representative value of each frame image becomes substantially equal, thereby ensuring consistency of the histogram. By this processing, the histogram of each frame image in the example shown in FIG. 3A eliminates the density difference between the frame images due to the performance of the imaging apparatus, and becomes as shown in FIG. In addition to the average value of the density values used here, a median, a variance value (maximum value, minimum value), or the like may be used as the representative value of the frame image.

  Next, the cut point likelihood calculation means 25 uses the histogram having consistency as shown in FIG. 3B, one by one before and after the target frame image (may be two or more). If the average shape of the histogram of the frame image is obtained to obtain an average shape, the correlation between the shape of the histogram of the frame image of interest and this average shape is obtained, and the obtained correlation is equal to or greater than a predetermined first threshold value, When the likelihood of the cut point of the frame image of interest is set to the minimum 0, and the correlation is equal to or less than a predetermined second threshold value (second threshold value <first threshold value), the likelihood of the cut point of the frame image of interest is set to the maximum 1 If the correlation is between the first threshold value and the second threshold value, the likelihood of the cut point of the frame image to be noticed between 0 and 1 is obtained so that the cut point probability decreases as the correlation increases. FIG. 3C shows the cut point likelihood β0 obtained for the six frame images in the example shown in FIG.

  The adjustment degree calculation means 30 uses the cut point likelihood β0 calculated by the cut point likelihood calculation means 25 to adjust the adjustment degree β for adjusting the temporary correction value γ0 according to the following equation (3) for each frame image. calculate.

β (i) = 1−β0 (i) (3)
However, β0: Like cut point
β: Degree of adjustment

The adding means 35 uses the adjustment degree β calculated by the adjustment degree calculation means 30 and the temporary correction value γ0 calculated by the temporary correction value calculation means 20, according to the following equation (4), A target correction value γ1 is calculated.

γ1 (i) = (1−β (i)) × γ0 (i)
+ Β (i) × (γ0 (i−1) + γ0 (i) + γ0 (i + 1)) / 3 (4)
Where γ1 (i): target correction value of the frame image of interest
γ0 (i): Temporary correction value of the frame image of interest
γ0 (i−1): provisional correction value of the frame image before the target frame image
γ0 (i + 1): provisional correction value of the frame image after the frame image of interest
β (i): Adjustment degree of the frame image of interest

The target correction value calculation unit 40 outputs the target correction value γ1 of each frame image obtained by the addition unit 35 to the correction unit 50.

  The correction means 50 corrects each frame image obtained by the image input means 1 according to the following equation (5) using the target correction value γ1 to obtain a corrected moving image.

YY = Y + γ1 (5)
YY: density value after correction
Y: Density value before correction
γ1: target correction value

Here, in order to make the gist of the present invention easy to understand, the target correction value calculation means 40 calculates the cut point likelihood β0, calculates the adjustment degree β based on the cut point likelihood β0, and uses the adjustment degree β. The target correction value γ1 is obtained by adjusting the temporary correction value γ0. For example, the target correction value γ1 is directly calculated by using the cut point likelihood β0 without calculating the adjustment degree β as shown in the following formula (6). The value γ1 may be obtained.

γ1 (i) = β0 (i)) × γ0 (i)
+ (1-β0 (i)) × (γ0 (i−1) + γ0 (i) + γ0 (i + 1)) / 3 (6)
However, γ1 (i): target correction value of the focused frame image γ0 (i): temporary correction value of the focused frame image γ0 (i−1): temporary correction value of the frame image before the focused frame image γ0 ( i + 1): Temporary correction value of the frame image after the frame image of interest β0 (i): Like cut point of the frame image of interest

FIG. 4 is a flowchart showing processing performed in the image processing apparatus A according to the embodiment shown in FIG. Note that the flowchart of FIG. 4 shows processing performed on one frame image constituting the moving image, and the image processing apparatus A of the present embodiment configures the processing shown in FIG. This is performed for each frame image to obtain a corrected moving image.

  As shown in FIG. 4, the frame image obtained by the image input means 1 is reduced by the reduction means 10 to become a reduced frame image (S10, S12). The temporary correction value calculation means 20 calculates a temporary correction value γ0 for exposure correction from the reduced frame image (S14). The target correction value calculation means 40 adjusts the temporary correction value γ0 and calculates the target correction value γ1 so that the temporary correction value γ0 of each frame image changes smoothly in the time direction (S20). The correcting unit 50 corrects the frame image obtained by the image input unit 1 using the target correction value γ1 (S40). The details of the target correction value calculation process P1 performed in step S20 by the target correction value calculation means 40 are shown in the flowchart of FIG. As shown in the figure, in the target correction value calculating means 40, first, the cut point likelihood calculating means 25 calculates the cut point likelihood β0 of the frame image of interest (S22), and then the adjustment degree calculating means 30 is the cut point likelihood. Based on β0, the adjustment degree β of the frame image of interest is calculated (S24). Then, the addition means 35 increases the weight of the temporary correction value of the frame image of interest (that is, the degree of adjustment of the temporary correction value of the frame image of interest) as the adjustment degree β is smaller (ie, the likelihood of cut point β0 is larger). The target correction value γ1 of the target frame image is weighted and added to the temporary correction value of the target frame image and the average value of the temporary correction values of the target frame image and the preceding and subsequent frame images. Calculate (S26).

  As described above, according to the image processing apparatus A of the present embodiment, in order to obtain the corrected moving image by correcting each frame image of the moving image, the temporary correction value obtained from each of the frame images is used as the time. When the target correction value is obtained by adjusting so as to change smoothly in the direction, the provisional correction value of this frame image increases as the cut point likelihood of the frame image of interest increases according to the cut point likelihood of each frame image. Pay attention by increasing the weight and reducing the weight of the average value of the temporary correction values of the frame image of interest and the preceding and succeeding frame images (that is, reducing the degree of adjustment of the temporary correction value of the frame image of interest). The target frame image and the average value of the temporary correction values of the frame image of interest and the frame images before and after the frame image to obtain the target correction value of the frame image of interest. Therefore, the target correction value of the frame image serving as the cut point (the frame image having the cut point likelihood of 1) can be prevented from being affected by the temporary correction values of the preceding and succeeding frame images. An appropriate correction value can be obtained. In addition, since the target correction value of a normal frame image (a frame image with a cut point likelihood of 0) having a small change in the same scene is affected by the temporary correction values of the preceding and subsequent frame images, the target correction value changes smoothly in the time direction. Flickers that occur in the corrected moving image can be eliminated. Furthermore, in the same scene as shown in FIG. 13, in the case of a frame image with a large image change (a frame image whose cut point likelihood is less than 1 greater than 0), the target correction value is the previous and next frames. While being influenced by the temporary correction value of the image, the degree of the influence is smaller than that of the normal frame image. Therefore, while correcting flicker in the corrected moving image, appropriate correction can be obtained even in a frame image that is in the same scene but has a large image change.

  FIG. 6 is a block diagram showing a configuration of an image processing apparatus B according to the second embodiment of the present invention.

As shown in FIG. 6, the image processing apparatus B of the present embodiment includes an image input unit 51 that acquires a plurality of frame images constituting a moving image, and a reduction unit 55 that reduces the frame image to obtain a reduced frame image. , A feature amount calculation unit 60 that calculates a feature amount of the reduced frame image, a temporary correction value calculation unit 65 that obtains a temporary correction value using the feature amount calculated by the feature amount calculation unit 60, and a temporary correction value calculation unit 65 The temporary correction means 70 for calculating the feature amount of the temporarily corrected reduced frame image obtained by correcting the reduced frame image using the temporary correction value obtained by the above, and the feature amount of the temporarily corrected reduced frame image. The target feature amount calculation unit 80 for obtaining the target feature amount of the reduced frame image and the feature amount calculated by the feature amount calculation unit 60 become the target feature amount calculated by the target feature amount calculation unit 80. As described above, the objective correction value calculation means 90 for obtaining the objective correction value of each frame image, and the correction of each frame image using the objective correction value obtained by the objective correction value calculation means 90 to obtain a corrected moving image Correction means 95 to be obtained.

  The feature amount calculating means 60 and the temporary correction value calculating means 65 correspond to the temporary correction value calculating means 20 of the image processing apparatus A shown in FIG. 1, and are reduced in the same manner as the temporary correction value calculating means 20 of the image processing apparatus A. A feature value Y0 of the frame image is obtained, and a correction value γ0 for correcting the exposure of the reduced frame image based on the feature value Y0 is obtained, and detailed description thereof is omitted here. Note that the thin line at the bottom of FIG. 8A indicates the feature amount Y0 calculated by the feature amount calculation means 60.

  The temporary correction value calculation unit 65 outputs the correction value γ0 to the temporary correction unit 70 as the temporary correction value of the reduced frame image.

  The temporary correction unit 70 uses the temporary correction value γ0 obtained by the temporary correction value calculation unit 65 and the feature amount Y0 obtained by the feature amount calculation unit 60 according to the following equation (7). A feature amount Y1 of the temporarily corrected reduced frame image obtained by correcting the reduced frame image using γ0 is obtained.

Y1 = Y0 + γ0 (7)
Y1: Feature value after temporary correction
Y0: feature value before temporary correction
γ0: Temporary correction value

Here, in order to increase the processing speed, the feature amount Y1 of the temporarily corrected reduced frame image is obtained directly without performing temporary correction on the reduced frame image. The feature amount of the temporarily corrected reduced frame image may be obtained after correction is performed to obtain the temporarily corrected reduced frame image. Specifically, exposure correction is performed on the reduced frame image according to the following equation (8) to obtain a temporarily corrected reduced frame image.

Y ′ = Y + γ0 (8)
Y ′: Density value after provisional correction
Y: Density value before temporary correction
γ0: Temporary correction value
Then, for the temporarily corrected reduced frame image, an intermediate value of the histogram of the density may be calculated in the same manner as the feature amount calculating means 60 to obtain the feature amount Y1 of the temporarily corrected reduced frame image.

  The thin line at the top in FIG. 8A indicates the feature amount Y1 of the temporarily corrected reduced frame image obtained by the temporary correction means 70. As illustrated, the feature amount Y1 of the temporarily corrected reduced frame image varies greatly in the time direction, and flickering occurs when a moving image composed of such a frame image is reproduced.

  The target feature amount calculation means 80 adjusts the feature amount of each temporarily corrected reduced frame image so as to obtain a target feature amount so that the feature amount of the temporarily corrected reduced frame image changes smoothly in the time direction. Here, as an example, attention is paid by performing a filling process (details will be described later) on the feature amount Y1 of the five temporarily corrected reduced frame images centered on the temporarily corrected reduced frame image of interest. The target feature amount Y2 is obtained by adjusting the feature amount of the temporarily corrected reduced frame image. As shown in FIG. 6, the target feature amount calculation means 80 in the present embodiment is, for each frame image, the likelihood of a cut point that is the magnitude of the probability that the frame image is a cut point indicating a scene change. Cut point likelihood calculation means 74 for calculating β0 and five temporary corrections centered on the temporarily corrected reduced frame image of interest with a filter determined according to the cut point likelihood β0 calculated by the cut point likelihood calculation means 74 And filtering means 76 for obtaining a target feature amount Y2 of the temporarily corrected reduced frame image to be noted by performing a filling process on the feature amount Y1 of the already reduced frame image.

  Like the cut point likelihood calculating unit 25 of the image processing apparatus A shown in FIG. 1, the cut point likelihood calculating unit 74 of the target feature amount calculating unit 80 is based on the shape of the histogram of the density value of the frame image. The detailed description thereof is omitted here. In the case of the example shown in FIG. 8A, the frame image corresponding to time t1 is a cut point indicating switching between scene 1 and scene 2, and the cut point likelihood is 1, corresponding to time t2. The frame image to be changed greatly changes in the scene 2, and the cut point likelihood is less than 1 and greater than 0.

  The filtering means 76 has a database (not shown) in which the cut point likelihood β0 is associated with the filter F used in accordance with the cut point likelihood, and FIG. 7 shows an example of this database. In addition, the arrow in the figure indicates the direction in which the likelihood of a cut point increases.

  The filtering unit 76 detects the filter F corresponding to the cut point likelihood β0 calculated by the cut point likelihood calculation unit 74 from the database shown in FIG. 7 and is calculated by the detected filter F and the temporary correction unit 70. The target feature amount Y2 is calculated according to the following equation (9) using the feature amount Y1 of the temporarily corrected reduced frame image.

Y2 (i) = F * (Y1 (i-2) + Y1 (i-1) + Y1 (i)
+ Y1 (i + 1) + Y1 (i + 2)) / 5 (9)
However, Y2 (i): target feature amount of the temporarily corrected reduced frame image to be noticed Y1 (i): feature amount of the temporarily corrected reduced frame image to be noticed Y1 (i-2): the temporarily corrected reduced frame image to be noticed Y1 (i-1): feature amount of the temporarily corrected reduced frame image immediately preceding the target temporarily corrected reduced frame image Y1 (i + 1): attention Feature amount of the temporarily corrected reduced frame image immediately after the temporarily corrected reduced frame image Y1 (i + 2): Feature amount of the temporarily corrected reduced frame image after the temporarily corrected reduced frame image of interest F: Cut Filter for pointiness

As can be seen from the filter of FIG. 7 and the above equation (9), as the cut point likelihood β0 of the temporarily corrected reduced frame image of interest increases, the target feature amount Y2 becomes larger than that of the temporarily corrected reduced frame images before and after that. In the case of a temporarily corrected reduced frame image in which the influence of the feature amount Y1 is small and the likelihood of a cut point is 1, the target feature amount Y2 is not affected by the preceding and following frame images. Further, as the cut point likelihood β0 of the temporarily corrected reduced frame image of interest is smaller, the target feature amount Y2 is more influenced by the feature amount Y1 of the temporarily corrected reduced frame images before and after that, and the cut point likelihood is 0. In the case of the temporarily corrected reduced frame image, the target feature amount Y2 is an average value of the feature amounts Y1 of the four temporarily corrected reduced frame images.

  The thick line in FIG. 8A indicates the target feature amount Y2 obtained for each reduced frame image by the target feature amount calculation means 80.

  The objective correction value calculating unit 90 is configured to use the following equation (10) so that the feature amount Y0 of the reduced frame image obtained by the feature amount calculating unit 60 becomes the target feature amount Y2 obtained by the target feature amount calculating unit 80. ) To obtain a correction value γ1, and outputs the correction value γ1 to the correction means 95 as a target correction value of the frame image.

γ1 = Y2−Y0 (10)
Y0: feature value before correction
Y2: target feature value
γ1: Target correction value

The correction means 95 corrects the frame image obtained by the image input means 51 according to the following equation (11) using the target correction value γ1 to obtain a corrected moving image.

YY = Y + γ1 (11)
YY: density value after correction
Y: Density value before correction
γ1: target correction value

FIG. 8B shows the result of correction by the image processing apparatus B of the present embodiment, and the thin line in the figure shows the feature quantity of each frame image constituting the moving image before correction. The middle thick line indicates the feature amount of each frame image constituting the corrected moving image. As illustrated, according to the image processing apparatus B of the present embodiment, first, in the corrected moving image, the corrected moving image is corrected so that the feature amount of each frame image in the same scene changes smoothly. The flicker of the image is surely eliminated. Further, for a frame image that is a cut point (a frame image corresponding to the time t1), (0, 0, 5, 0, 0) is used as the filter F when obtaining the target feature amount. Since it is not affected by the feature values of the preceding and subsequent temporarily corrected reduced frame images, appropriate correction is performed using the correction value obtained based on the target feature value. Further, in the case of a frame image that is the same scene as the frame image corresponding to the time t2 but has a large image change (a frame image in which the likelihood of a cut point is less than 1 that is greater than 0), the target feature amount is While being affected by the feature amount of the temporarily corrected reduced frame images before and after, the degree of the influence is smaller than that of the normal frame image, so the flicker in the corrected moving image is eliminated and the image is the same scene. Appropriate correction can be obtained even in a frame image with a large change in.

  Further, the image processing apparatus B obtains a temporary correction value for a plurality of frame images constituting the moving image, and the feature amount of the temporarily corrected frame image obtained by performing the temporary correction using the temporary correction value is in the time direction. The value obtained by adjusting the feature value of each temporarily corrected frame so as to change smoothly is used as the target feature value, and the target correction of each frame image is performed so that the feature value of the frame image becomes substantially equal to the target feature value. The target correction value is obtained by directly suppressing the transition of the target feature amount, that is, the feature amount of the corrected frame image, while taking advantage of the image quality improvement effect of the temporary correction. Therefore, the flicker of the corrected moving image can be resolved more reliably than the image processing apparatus A shown in FIG. 1 that suppresses only the transition of the temporary correction value.

  Next, processing performed in the image processing apparatus B of the present embodiment will be described with reference to the flowchart of FIG. Note that the flowchart in FIG. 9 shows processing performed on one frame image constituting the moving image, and the image processing apparatus B of the present embodiment configures the processing shown in FIG. 9 in the moving image. This is performed for each frame image to obtain a corrected moving image.

  As shown in FIG. 9, the frame image obtained by the image input means 51 is reduced by the reduction means 55 to become a reduced frame image (S51, S52). The feature amount calculating means 60 calculates a feature amount Y0 from the reduced frame image (S54). The temporary correction value calculating unit 65 calculates a temporary correction value γ0 for exposure correction based on the feature amount Y0 (S56), and the temporary correction unit 70 calculates the temporary correction value γ0 and the feature amount calculated in step S54. The feature amount Y1 of the temporarily corrected reduced frame image is calculated using Y0 (S58). The target feature amount calculation unit 80 adjusts the feature amount Y1 so that the feature amount Y1 changes smoothly in the time direction to obtain the target feature amount Y2 (S60). The objective correction value calculation means 90 calculates the objective correction value γ1 so that the feature quantity Y0 calculated in step S54 becomes the target feature quantity Y2 calculated in step S60 (S70). The correcting unit 95 corrects the frame image obtained by the image input unit 51 using the target correction value γ1 (S72). The details of the target feature amount calculation process P2 performed in step S60 by the target feature amount calculation means 80 are shown in the flowchart of FIG. As shown in the figure, in the target feature amount calculation means 80, first, the cut point likelihood calculation means 74 calculates the cut point likelihood β0 of the frame image of interest (S62). Next, the filtering unit 76 selects a filter corresponding to the cut point likelihood β0 and sets the feature amount Y1 (feature amount of the temporarily corrected reduced frame image) of five consecutive frame images centered on the frame image of interest. On the other hand, by applying the selected filter, the target feature amount Y2 of the target frame image is calculated (S64, S66). Note that the filter selected in step S64 is such that, as shown in FIG. 7, the extent to which the target feature amount is influenced by the preceding and succeeding frame images becomes smaller as the cut-point likelihood β0 increases. is there.

  FIG. 11 is a block diagram showing a configuration of an image processing apparatus C according to the third embodiment of the present invention. As illustrated, the image processing apparatus C according to the present embodiment includes an image input unit 101 that acquires a plurality of frame images constituting a moving image, a reduction unit 110 that reduces a frame image to obtain a reduced frame image, and a reduction unit. A feature amount calculation unit 115 that calculates a feature amount of a frame image, a temporary correction value calculation unit 120 that calculates a temporary correction value using the feature amount calculated by the feature amount calculation unit 115, and a temporary correction value calculation unit 120 Temporary correction means 130 for calculating a feature amount of the temporarily corrected reduced frame image obtained by correcting the reduced frame image using the provisional correction value obtained, and each reduced frame using the feature amount of the temporarily corrected reduced frame image The target feature quantity calculation means 140 for obtaining the target feature quantity of the image and the feature quantity calculated by the feature quantity calculation means 115 are obtained by the target feature quantity calculation means 140. Objective correction value calculation means 150 for obtaining an objective correction value for each frame image so as to obtain a target feature amount, and correction for each frame image using the objective correction value obtained by the objective correction value calculation means 150 Correction means 160 for obtaining a corrected moving image. Note that the image processing apparatus C of the present embodiment differs from the image processing apparatus B of the embodiment shown in FIG. 5 in all other configurations except that the target feature quantity calculation unit 140 and the target feature quantity calculation unit 80 are different. Therefore, only the processing performed in the target feature amount calculation unit 140 will be described here, and detailed description of other configurations will be omitted.

  The target feature amount calculation unit 140 of the image processing apparatus C adjusts the feature amount of each temporarily corrected reduced frame image so that the feature amount of the temporarily corrected reduced frame image changes smoothly in the time direction, thereby obtaining the target feature amount. As shown in FIG. 11, for each frame image, the likelihood of a cut point for calculating the cut point probability β0, which is the magnitude of the probability that the frame image is a cut point indicating a scene change. A temporary target feature amount calculating unit 144 that obtains a temporary target feature amount Y1a by smoothing the feature amount Y1 of each temporarily corrected reduced frame image obtained by the calculating unit 142, the temporary correction unit 130, and the temporarily corrected target of interest The feature amount Y1 and the temporary target feature amount Y1a of the temporarily corrected reduced frame image of interest are weighted and added with a weight according to the cut point likelihood β0 of the reduced frame image. And adding means 146 for obtaining the collection amount Y2.

  Like the cut point likelihood calculation unit 74 of the image processing apparatus B shown in FIG. 5, the cut point likelihood calculation unit 142 calculates the cut point likelihood based on the shape of the histogram of the density values of the frame image. Here, detailed description thereof is omitted.

  The temporary target feature amount calculation unit 144 has five temporarily centered on the temporarily corrected reduced frame image to be noticed (from the temporarily corrected reduced frame image immediately before the temporarily corrected reduced frame image to be noticed. The value obtained by averaging the feature amounts Y1 of the temporarily corrected reduced frame images after the reduced frame image (5 up to the second temporarily corrected reduced frame image) is the temporarily corrected reduced frame image to which attention is paid. The temporary target feature amount Y1a.

  According to the following equation (12), the adding unit 146 increases the weight of the feature amount Y1 and decreases the weight of the temporary target feature amount Y1a as the cut point likelihood β0 of the temporarily corrected reduced frame image of interest increases. Then, the target feature amount Y2 of the temporarily corrected reduced frame image of interest is obtained by weighting and adding the feature amount Y1 and the provisional target feature amount Y1a.

Y2 = β0 × Y1 + (1−β0) × Y1a (12)
Y2: target feature amount of the temporarily corrected reduced frame image of interest Y1: feature amount of the temporarily corrected reduced frame image of interest Y1a: temporary target feature amount β0: likelihood of a cut point of the temporarily corrected reduced frame image of interest

FIG. 12 is a flowchart showing processing performed in the target feature amount calculation unit 140 in the image processing apparatus C shown in FIG. As shown in the figure, the target feature amount calculation unit 140 of the image processing apparatus C calculates the cut point likelihood β0 of the frame image of interest by the cut point likelihood calculation unit 142 (S100) and the temporary target feature amount calculation unit 144. Thus, five temporarily corrected reduced frame images from the temporarily corrected reduced frame image two before the temporarily corrected frame image of interest to the temporarily corrected reduced frame image of the second after the temporarily corrected frame image of interest. By calculating the average of the feature amounts Y1, the temporary target feature amount Y1a of the temporarily corrected reduced frame image to be noticed is obtained (S110). Then, the target feature value Y2 is obtained by weighting and adding the feature value Y1 of the temporarily corrected reduced frame image of interest and its temporary target feature value Y1a by the adding means 146 (S120). The weighting addition in step S120 by the adding unit 146 is performed such that the weight of the feature amount Y1 is larger and the weight of the temporary target feature amount Y1a is smaller as the cut point likelihood β0 is larger.

  Although the image processing device C of the present embodiment is different from the image processing device B in the calculation method of the target feature amount, the target feature amount is affected by the preceding and succeeding frame images as the frame image is more likely to be cut. Since it is required to be small, the same effect as the image processing apparatus B can be obtained. Furthermore, since the weights of the feature amount Y1 and the temporary target feature amount Y1a are adjusted according to the cut point likelihood, the processing is simple.

    The preferred embodiments of the present invention have been described above. However, the image processing method and apparatus of the present invention and the program therefor are not limited to the above-described embodiments, and each of the above-described embodiments is not departed from the gist of the present invention. The configuration of the embodiment can be combined, increased, decreased, or changed.

  For example, in the image processing apparatus B and the image processing apparatus C, the processing from the calculation of the temporary correction value to the calculation of the target feature amount is performed on each frame image constituting the moving image. A reference frame image is selected from the frame images constituting the moving image, processing is performed until the target feature amount is calculated only for the selected reference frame image, and spline interpolation is performed on the target feature amount of the reference frame image. Alternatively, the target feature amount of another frame image may be obtained by performing interpolation processing such as linear interpolation. By doing so, processing such as calculation of the temporary correction value and calculation of the feature amount of the temporarily corrected reduced frame image is performed only on the reference frame image, and the processing speed can be increased. The selection of the reference frame image can be performed, for example, by obtaining the similarity between the frame images in the time direction and determining the interval between the reference frame images. Here, as the similarity, for example, the correlation between the frame images can be used so that the higher the correlation is, the higher the similarity is. Also, if the determined interval between reference frame images is n (n: an integer equal to or greater than 2), each of the frame images (frame image 1, frame image 2,...) Constituting the moving image. Frame image 1, frame image (n + 1), frame image (2n + 1), frame image (3n + 1). . . The frame image may be a reference frame image.

  In the image processing apparatus according to the above-described embodiment, the processing from the calculation of the feature amount to the target correction value is performed on the reduced frame image so as to increase the processing speed. However, the frame image is not reduced. Alternatively, these processes may be performed. In the case where the reduced frame image is reduced and processed, the reduction ratio may be adjusted according to the required processing time and the resolution of the original frame image.

  In the image processing apparatus according to the above-described embodiment, the intermediate value Y0 of the density histogram is used as the feature amount of the frame image. However, the average value of the density in the frame image, a representative value such as a median, and the like are used as the feature amount. You may do it.

  Further, brightness or the like may be used instead of the density used in the above-described embodiment.

  The image processing apparatus described above performs exposure correction on a moving image as an example. However, the image processing method and apparatus of the present invention are not limited to exposure correction, but include the above-described white balance correction and saturation correction. The present invention can also be applied to other image quality corrections such as contrast correction, sharpness correction, and color correction, and combinations of these image quality corrections.

  In addition, the image processing apparatus described above performs processing until correction is performed using the target correction value obtained for the frame image, but the target correction value is applied to the original moving image without performing the correction. Correction may be executed when an instruction is given by attaching it to a recording medium or transmitting it to a processing device on a network.

1 is a block diagram showing a configuration of an image processing apparatus A according to a first embodiment of the present invention. The figure for demonstrating the feature-value of a frame image Diagram for explaining the calculation of cut point likelihood Flowchart showing processing performed in image processing apparatus A (part 1) Flowchart showing processing performed in image processing apparatus A (part 2) The block diagram which shows the structure of the image processing apparatus B used as the 2nd Embodiment of this invention. The figure which shows the relationship between cut point likeness and the filter F The figure for demonstrating operation | movement of the image processing apparatus B Flow chart showing processing performed in image processing apparatus B (part 1) Flowchart showing processing performed in image processing apparatus B (part 2) The block diagram which shows the structure of the image processing apparatus C used as the 3rd Embodiment of this invention. The flowchart which shows the process performed in the target feature-value calculation means 140 in the image processing apparatus C The figure which shows the example of the frame image with the big fluctuation in the same scene

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image input means 10 Reduction means 20 Temporary correction value calculation means 25 Cut point likelihood calculation means 30 Adjustment degree calculation means 35 Addition means 40 Objective correction value calculation means 50 Correction means 51 Image input means 55 Reduction means 60 Feature amount calculation means 65 Temporary Correction value calculation means 70 Temporary correction means 74 Cut point likelihood calculation means 76 Filtering means 80 Target feature amount calculation means 90 Target correction value calculation means 95 Correction means 101 Image input means 110 Reduction means 115 Feature quantity calculation means 120 Temporary correction value calculation means 130 Temporary correction means 140 Target feature amount calculation means 142 Cut point likelihood calculation means 144 Temporary target feature amount calculation means 146 Addition means 150 Target correction value calculation means 160 Correction means β0 Cut point likelihood γ0 Temporary correction value γ1 Target correction value

Claims (12)

Calculating a feature amount of each frame image constituting a moving image, a likeness cut point indicating the magnitude of the probability is a cut point of the frame image showing the switching comparatively scene in the moving image,
Obtaining a temporary correction value for correcting the image quality of the frame image;
Correct each frame image using the temporary correction value ,
Calculate the feature value of each temporarily corrected frame image after correction,
By performing a smoothing process in the time direction on the feature amount of each provisionally corrected frame image so that the frame image having a higher likelihood of the cut point has a lower smoothness level that smoothes the change in the feature amount. obtains a material obtained by adjusting the characteristics of each temporarily corrected frame image as the target feature amount,
Each frame image is corrected so that the feature amount of the frame image is substantially equal to the target feature amount of the frame image.
An image processing method. The feature amount of each frame image constituting the moving image and the cut point likelihood indicating the probability that the frame image is a cut point indicating scene switching in the moving image are calculated for each frame image. And
A temporary correction value for correcting the image quality of the frame image is obtained from each frame image,
Correct each frame image using the temporary correction value ,
Calculate the feature value of each temporarily corrected frame image after correction,
Applying a smoothing process in the time direction to the calculated feature amount of each provisionally corrected frame image so that the smoothness level for smoothing the change in the feature amount becomes weaker as the frame image has a higher likelihood of the cut point. To obtain a temporary target feature value obtained by adjusting the feature value of each provisionally corrected frame image,
Obtaining a value obtained by weighted addition of the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image as the target feature amount of the frame image ;
An image processing method , wherein each frame image is corrected so that a feature amount of the frame image is substantially equal to the target feature amount of the frame image . The target feature amount is calculated only for the reference frame image selected from each frame image constituting the moving image,
The image processing method according to claim 1 or 2, wherein the obtaining the target characteristic quantity of each frame image constituting the moving image by performing an interpolation process with respect to the target characteristic quantity of the reference frame picture . Any of claims 1 to 3, which comprises carrying out the process of using the reduced images of the frame images constituting the moving image from the calculation of the feature amount of the frame image to obtain the above object correction value 1 An image processing method according to item. A feature amount calculating means for calculating a feature amount of each frame image of a plurality of frame images constituting the moving image;
A likeness cut point calculation means for obtaining relative to the frame image is the probability a cut point showing the switching comparatively scene in the video size each frame image likeness cut point indicating a
Target feature amount calculating means for obtaining a target feature amount that is a target value of the feature amount of each frame image;
Objective correction value calculation means for obtaining a previous objective correction value such that the feature quantity of the frame image is substantially equal to the target feature quantity of the frame image;
Correction means for correcting the frame image using the target correction value;
Have
The target feature amount calculating means includes
Provisional correction value calculation means for obtaining a provisional correction value for correcting the image quality of the frame image from each frame image;
Temporary correction means for calculating a feature amount of a temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
Applying a smoothing process in the time direction to the calculated feature amount of each provisionally corrected frame image so that the smoothness level for smoothing the change in the feature amount becomes weaker as the frame image has a higher likelihood of the cut point. Accordingly, the each temporarily corrected frame image feature amount smoothing means Ru calculated as the target feature amount obtained by adjusting the characteristics of
The image processing apparatus characterized by comprising a. A feature amount calculating means for calculating a feature amount of each frame image of a plurality of frame images constituting the moving image;
A likeness cut point calculation means for obtaining relative to the frame image is the probability a cut point showing the switching comparatively scene in the video size each frame image likeness cut point indicating a
Target feature amount calculating means for obtaining a target feature amount that is a target value of the feature amount of the frame image;
Objective correction value calculation means for obtaining a previous objective correction value such that the feature quantity of the frame image is substantially equal to the target feature quantity of the frame image;
Correction means for correcting the frame image using the target correction value;
Have
The target feature amount calculating means includes
Provisional correction value calculation means for obtaining a provisional correction value for correcting the image quality of the frame image from each frame image;
Temporary correction means for calculating a feature amount of a temporarily corrected frame image obtained by correcting each frame image using the temporary correction value;
Applying a smoothing process in the time direction to the calculated feature amount of each provisionally corrected frame image so that the smoothness level for smoothing the change in the feature amount becomes weaker as the frame image has a higher likelihood of the cut point. The feature amount smoothing means for obtaining the provisional target feature amount by adjusting the feature amount of each provisionally corrected frame image,
Weighted addition means for obtaining a value obtained by weighted addition of the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image as the target feature amount of the frame image;
An image processing apparatus comprising: The target feature amount calculating means calculates the target feature amount only for a reference frame image selected from among the respective frame images constituting the moving image, and is obtained using the reference frame image. by performing the interpolation process on the target feature amount obtained each according to claim 5 or 6 characterized in that it has an interpolation means for obtaining the target characteristic quantity of each of the frame images constituting the moving picture Image processing device. Each unit, the image processing apparatus according to any one of claims 5, wherein 7 that performs the processing of each using a reduced image of frame images forming the moving image. On the computer,
  Calculating the feature amount of each frame image constituting the moving image and the cut point likelihood indicating the probability that the frame image is a cut point indicating scene switching in the moving image;
  Obtaining a temporary correction value for correcting the image quality of the frame image;
  Correct each frame image using the temporary correction value,
  Calculate the feature value of each temporarily corrected frame image after correction,
Applying a smoothing process in the time direction to the calculated feature amount of each provisionally corrected frame image so that the smoothness level for smoothing the change in the feature amount becomes weaker as the frame image has a higher likelihood of the cut point. To obtain a target feature value obtained by adjusting the feature value of each provisionally corrected frame image,
  Each frame image is corrected so that the feature amount of the frame image is substantially equal to the target feature amount of the frame image.
  A program to make things happen.   On the computer,
  The feature amount of each frame image constituting the moving image and the cut point likelihood indicating the probability that the frame image is a cut point indicating scene switching in the moving image are calculated for each frame image. And
  A temporary correction value for correcting the image quality of the frame image is obtained from each frame image,
  Correct each frame image using the temporary correction value,
  Calculate the feature value of each temporarily corrected frame image after correction,
  In order that the calculated feature amount of each of the temporarily corrected frame images changes smoothly in the time direction, the smoothness degree that smoothes the change in the feature amount is weakened as the frame image having a higher likelihood of the cut point is obtained. Obtained as a temporary target feature amount of the frame image the adjustment of the feature amount of each temporarily corrected frame image,
  Obtaining a value obtained by weighted addition of the feature amount of the provisionally corrected frame image and the provisional target feature amount of the frame image as the target feature amount of the frame image;
  Each frame image is corrected so that the feature amount of the frame image is substantially equal to the target feature amount of the frame image.
  A program to make things happen.   The target feature amount is calculated only for the reference frame image selected from each frame image constituting the moving image,
  The program according to claim 9 or 10, wherein the target feature amount of each frame image constituting the moving image is obtained by performing an interpolation process on the target feature amount of the reference frame image.   The process from the calculation of the feature amount of the frame image to the determination of the target correction value is performed using a reduced image of the frame image constituting the moving image. Program described in the section.

Images