JP5211864B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus.

Statistical analysis is performed on a plurality of frames included in a predetermined time interval in a moving image, and a conversion characteristic for performing gradation value conversion is determined based on the analysis result, and a predetermined time is determined according to the conversion characteristic. There is known an image processing apparatus that performs gradation value conversion of each pixel in each frame included in a section (see, for example, Patent Document 1).
JP 2000-078619 A

  However, the above-described conventional image processing apparatus can stably output the hue and lightness even if the shooting conditions change, but more naturally reproduces the gradation of the subject in a specific area in the image. There is a problem that can not be.

(1) The invention of claim 1 is an image processing apparatus that performs gradation correction by acquiring a plurality of continuous frame images, and an area setting means for setting a correction target area in a frame image to be corrected. And extracting at least one frame image as a reference frame image from frame images before and after the frame image to be subjected to gradation correction, and using the image in the correction target region as a template image from the reference frame image. Area detection means for detecting a corresponding area, which is an image area corresponding to the image of the correction target area, by template matching ; brightness information of the correction target area in at least the frame image of the gradation correction target; and the reference frame image the luminance information of the corresponding area, and based on the histogram of the gradation conversion curve and the luminance distribution before the correction target frame in With a tone conversion curve determination means for determining a gradation conversion curve of the current correction target frame, and a gradation correction means for performing gradation correction of the gradation correction target frame image by the gradation conversion curve, The gradation conversion curve determining means loosens the conversion characteristics of the gradation conversion curve as it moves away from the correction target area outside the correction target area in the gradation correction target frame image .
(2) The invention of claim 2 is an image processing apparatus that performs gradation correction by acquiring a plurality of continuous frame images, and an area setting means for setting a correction target area in a frame image to be corrected. And extracting at least one frame image as a reference frame image from frame images before and after the frame image to be subjected to gradation correction, and using the image in the correction target region as a template image from the reference frame image. Area detection means for detecting a corresponding area, which is an image area corresponding to the image of the correction target area, by template matching ; brightness information of the correction target area in at least the frame image of the gradation correction target; and the reference frame image the luminance information of the corresponding area, and based on the histogram of the gradation conversion curve and the luminance distribution before the correction target frame in With a tone conversion curve determination means for determining a gradation conversion curve of the current correction target frame, and a gradation correction means for performing gradation correction of the gradation correction target frame image by the gradation conversion curve, When the time between the plurality of consecutive frame images is short, a new gradation conversion curve is determined for each predetermined frame, and the previous gradation conversion curve is used for the other frames .
(3) The invention of claim 3 is the image processing apparatus according to claim 1 or 2 , wherein the region detecting means increases the luminance change in a plurality of continuous frame images constituting the moving image. A large number of the reference frame images are extracted.
(4) The invention of claim 4 is the image processing apparatus according to any one of claims 1 to 3 , wherein the area detection unit is configured to detect a plurality of frame images continuous in a time series constituting the moving image. The larger the movement of the correction target area, the more reference frame images are extracted.
(5) The invention of claim 5 is the image processing apparatus according to any one of claims 1 to 4 , wherein the region setting unit is configured such that a user operates an operation member in the frame image to be subjected to gradation correction. The area designated by using the correction area is set as the correction target area.
(6) The invention of claim 6 is the image processing apparatus according to any one of claims 1 to 4 , wherein the region setting means recognizes a human face portion in the frame image to be subjected to gradation correction. Then, an area including the portion is set as the correction target area.

  According to the present invention, the gradation of a subject in a region of interest in an image can be reproduced more naturally and stably.

  An embodiment in which the image processing apparatus and the image processing method of the present invention are applied to a camera capable of recording and reproducing a moving image such as a digital video camera or a digital camera will be described. Note that the present invention can be applied to a digital video camera and a moving picture taken by a camera other than the digital camera, or a series of still pictures taken continuously in time series.

  FIG. 1 shows the configuration of an image processing apparatus according to an embodiment. In this embodiment, an example in which a moving image captured and recorded by the camera 1 is subjected to gradation correction by the image processing device 2 is shown. Note that the present invention is not limited to this, and an image to be processed may be input from a recording apparatus instead of being input from a camera. The camera 1 includes a recording / reproducing unit 1a for recording or reproducing a moving image, a monitor 1b for displaying the moving image, and the like. The image processing device 2 stores an A / D, a D / A conversion unit 2a for converting an analog image signal and a digital image signal, a microcomputer 2b for performing gradation correction by performing various processes on the moving image, and a moving image frame. A memory 2c for storing moving images, a storage device 2d for storing moving images, an operation unit 2e for performing various operations, a monitor 2f for displaying images, and the like.

  The microcomputer 2b of the image processing device 2 captures the moving image captured by the camera 1 as an analog image signal for each frame, converts it to a digital image signal by the A / D, D / A conversion unit 2a, and then stores it in the storage device 2d. Memorize temporarily. Next, a series of frame images constituting the moving image is transferred to the memory 2c, and developed on the memory 2c in units of pixels represented by RGB three primary color components. Then, a correction process for more naturally reproducing the gradation of the specific area for each frame image is performed on a plurality of continuous frame images within a predetermined time.

  FIG. 2 is a flowchart showing the tone correction processing of the image processing apparatus 2. In step 1, the microcomputer 2b of the image processing apparatus 2 extracts a frame image to be processed from a plurality of consecutive frame images developed within the predetermined time on the memory 2c, and displays the extracted frame image on the monitor 2f. This specifies which frame image in a series of multiple frame images developed in time series is to be processed, and for a series of multiple frame images within a certain period of time. When processing is performed, it is a frame for starting processing, and when processing is performed on all frames of a moving image, it is the first frame. If this gradation correction processing has already been started, the next frame in the time series is selected as the processing target frame.

  In step 2, an area for which gradation is to be corrected is set from the frame image to be processed displayed on the monitor 2f, and is set as a correction target area. The correction target area may be set by the user manually specifying the intended main subject by moving the cursor with the direction key of the operation unit 2e, or automatically detecting, for example, a human face area from the image. It may be set with. Note that the correction target area is not necessarily limited to one, but may be a plurality. Correction is performed so that the image in the correction target area set here has the best image quality and gradation by the following processing.

  In the subsequent step 3, frames before and after referring to the correction of the gradation of the correction target frame (hereinafter referred to as a reference frame) are selected. In this embodiment, the gradation of the correction target area in the correction target frame is corrected with reference to the luminance information of the frames before and after the correction target frame. Several frames before and after the correction target frame are selected as reference frames. However, if a scene that does not correlate with the correction target frame exists in the previous or next frame due to a change in the imaging scene, such a frame is not selected as the reference frame, and in such a case, the frame before the correction target frame is selected. Or only subsequent frames.

  In step 4, a region corresponding to the correction target region of the correction target frame (hereinafter referred to as a corresponding region) is detected from the reference frame. Specifically, the image of the correction target region in the correction target frame is used as a template image, template matching for detecting an image region similar to the template image from the reference frame image is performed, and the similar image region is set as a corresponding region.

  In step 5, a gradation conversion curve for the correction target frame is determined based on the luminance information of the corresponding region in the reference frame image. Here, an example in which gradation correction is performed by a histogram equalization method having a contrast enhancement effect is shown. The histogram equalization method is a method of uniformly distributing the luminance signal distribution, and is a process of converting an image with low contrast into an image with clear shading. In step 6, gradation correction of the correction target frame image is performed using the gradation conversion curve.

  FIG. 3 is a diagram showing an example of gradation correction by the histogram equalization method. When a histogram of the number of pixels with respect to the luminance value for each pixel of the original image shown in (a) is measured, a histogram with a narrow luminance distribution range a as shown in (b) is obtained. When tone correction is performed on such an original image with a narrow luminance distribution by the histogram equalization method, an image with enhanced contrast as shown in (c) is obtained, and the histogram is as shown in (d). In addition, the luminance distribution range b is widened.

  As a specific correction method of the histogram equalization method, a gradation conversion curve as shown in FIG. 4 is determined and applied to the original image, that is, the frame image to be corrected. In FIG. 4, the horizontal axis represents the luminance value of the input pixel, and the vertical axis represents the luminance value of the output pixel. 4 for converting an original image having a narrow luminance distribution range a as shown in FIGS. 3A and 3B into an image having a wide luminance distribution range b as shown in FIGS. 3C and 3D. Is determined, and the output value of the intersection with the gradation conversion curve is obtained using the luminance value for each pixel of the correction target frame image as an input value. When this conversion processing is performed on all the pixels of the correction target frame image, images having low contrast as shown in FIGS. 3 (a) and 3 (b) are used as shown in FIGS. 3 (c) and 3 (d). An image with enhanced contrast is obtained.

  When determining the gradation conversion curve, as shown in FIG. 5, the luminance information of the correction target area of the correction target frame, the luminance information of the corresponding area of the previous and subsequent frames of the correction target frame, and the gradation conversion of the previous correction target frame Based on the curve and the histogram, the gradation conversion curve of the correction target frame is determined.

  In the above-described histogram equalization method, the gradation conversion curve is determined based on the luminance information in the correction target area of the correction target frame and the luminance information in the corresponding area of the reference frame, and the gradation correction processing is performed. However, if a gradation conversion curve determined based on local luminance information in the image is applied to the entire image, the balance of the entire image may be lost. Therefore, by making the gradation conversion curve gentle as the distance from the correction target area of the correction target frame increases, it is possible to reduce a large disturbance in the entire image.

  In step 7, when the gradation correction processing is sequentially performed on a plurality of consecutive frame images within a predetermined time, the process returns to step 1 and the above-described gradation correction processing is performed on the next frame image in time series. Apply. If the time interval between frames on the time series in the moving image is short, the process returns from step 7 to step 1 every predetermined number of frames to newly determine the gradation conversion curve and perform gradation correction. For other frames, the process returns from step 7 to step 5 to perform tone correction using the previously obtained tone conversion curve.

  In the above-described embodiments and their modifications, all combinations of the embodiments and the modifications are possible.

  According to the above-described embodiment and its modifications, the following operational effects can be achieved. First, when performing gradation correction on a plurality of continuous frame images constituting a moving image, a correction target area is set in the frame image to be corrected and the frame to be corrected. At least one frame image is extracted as a reference frame image from frame images before and after the image, and an image region similar to the image of the correction target region is detected as a corresponding region from the reference frame image. Then, based on at least the luminance information of the correction target area in the gradation correction target frame image and the luminance information of the corresponding area in the reference frame image, the image quality of the correction target area in the gradation correction target frame image is determined. The gradation conversion curve to be improved is determined, and the gradation correction of the frame image to be corrected is performed using the gradation conversion curve, so that the gradation of the main subject in the region of interest in the image is more naturally stable. And can be reproduced.

  In addition, since the gradation conversion curve determined in the previous frame image to be corrected and the histogram of the luminance distribution are considered, the gradation conversion curve of the frame image to be corrected this time is determined. The gradation of the main subject in the attention area can be reproduced more stably.

  In addition, since the conversion characteristics of the gradation conversion curve are made gentler as the distance from the correction target area increases outside the correction target area in the frame image to be corrected for gradation, the conversion characteristics of the gradation conversion curve become local to the attention area in the image. Even if the gradation correction is performed in general, the large disturbance of the entire image can be reduced, and the gradation of the entire image as well as the attention area can be reproduced naturally.

  Furthermore, since the larger the number of reference frame images is extracted, the greater the change in brightness in the multiple time-series frames that make up the video, the more stable the gradation of the main subject in the region of interest. Can be reproduced.

  Next, since the larger the number of reference frame images is extracted, the larger the movement of the correction target area in a plurality of frame images that are continuous in time series constituting the moving image, the gradation of the main subject in the attention area is adjusted. It can be reproduced more stably.

  In addition, since the area specified by the user using the operation member in the frame image to be corrected for gradation is set as the correction target area, the user can arbitrarily select the main subject whose gradation is to be corrected and its range. The gradation of the intended main subject can be reproduced more naturally.

  In addition, if the face portion of a person is recognized in the frame image to be subjected to gradation correction, and the region including the portion is set as the correction target region, the gradation of the person's face portion is automatically made more natural. Can be reproduced.

1 is a diagram illustrating a configuration of an image processing apparatus according to an embodiment. The flowchart which shows operation | movement of the image processing apparatus of one embodiment. The figure which shows the example of gradation correction by the histogram equalization method Diagram showing an example of a gradation conversion curve The figure which shows the concept which performs gradation correction | amendment with respect to the several frame image continuous on the time series

Explanation of symbols

2; Image processing device, 2b; Microcomputer (microcomputer), 2c; Memory, 2d; Storage device, 2e;

Claims (6)

An image processing apparatus that acquires a plurality of continuous frame images and performs gradation correction,
Area setting means for setting a correction target area in a frame image to be corrected for gradation;
At least one frame image is extracted as a reference frame image from frame images before and after the frame image to be subjected to gradation correction, and the correction is performed from the reference frame image by using the image of the correction target region as a template image. Area detecting means for detecting a corresponding area, which is an image area corresponding to the image of the target area, by template matching ;
At least the luminance information of the correction target area in the frame image to be corrected for gradation, the luminance information of the corresponding area in the reference frame image , and the gradation conversion curve and luminance distribution histogram of the previous correction target frame And a gradation conversion curve determining means for determining a gradation conversion curve of the current correction target frame based on
Gradation correction means for performing gradation correction of the frame image to be corrected by the gradation conversion curve ,
The gradation conversion curve determining means loosens the conversion characteristics of the gradation conversion curve as the distance from the correction target area increases, outside the correction target area in the gradation correction target frame image. An image processing apparatus. An image processing apparatus that acquires a plurality of continuous frame images and performs gradation correction,
Area setting means for setting a correction target area in a frame image to be corrected for gradation;
At least one frame image is extracted as a reference frame image from frame images before and after the frame image to be subjected to gradation correction, and the correction is performed from the reference frame image by using the image of the correction target region as a template image. Area detecting means for detecting a corresponding area, which is an image area corresponding to the image of the target area, by template matching ;
At least the luminance information of the correction target area in the frame image to be corrected for gradation, the luminance information of the corresponding area in the reference frame image , and the gradation conversion curve and luminance distribution histogram of the previous correction target frame And a gradation conversion curve determining means for determining a gradation conversion curve of the current correction target frame based on
Gradation correction means for performing gradation correction of the frame image to be corrected by the gradation conversion curve,
When the time between the plurality of consecutive frame images is short, a new gradation conversion curve is determined for each predetermined frame, and the previous gradation conversion curve is used for other frames. An image processing apparatus. The image processing apparatus according to claim 1 or 2 ,
The image processing apparatus is characterized in that the area detecting means extracts a larger number of the reference frame images as the luminance change in a plurality of continuous frame images on the time series constituting the moving image is larger. In the image processing device according to any one of claims 1 to 3 ,
The image processing apparatus, wherein the area detecting unit extracts a larger number of the reference frame images as the movement of the correction target area increases in a plurality of frame images continuous in time series constituting the moving image. . In the image processing device according to any one of claims 1 to 4 ,
The image processing apparatus according to claim 1, wherein the region setting means sets a region specified by a user using an operation member in the gradation correction target frame image as the correction target region. In the image processing device according to any one of claims 1 to 4 ,
The image processing apparatus according to claim 1, wherein the region setting unit recognizes a human face portion in the frame image to be subjected to gradation correction, and sets a region including the portion as the correction target region.

Images