WO2007142048A1

WO2007142048A1 - Video signal processing device, video signal processing method, and video signal processing program

Info

Publication number: WO2007142048A1
Application number: PCT/JP2007/060758
Authority: WO
Inventors: Hiroshi Suzuki
Original assignee: Olympus Corporation
Priority date: 2006-06-07
Filing date: 2007-05-22
Publication date: 2007-12-13
Also published as: JP2007329619A

Abstract

A video signal imaged via a CCD (102) is transferred via an A/D (104) to a buffer (105). The video signal in the buffer (105) is transferred to a light measurement evaluating unit (106), a distance information calculation unit (108), and a signal processing unit (110). The distance information calculation unit (108) calculates distance information on a distance between an imaging device and an object and transfers the distance information to a lens control unit (107) and a correction coefficient calculation unit (112). The correction coefficient calculation unit (112) calculates a correction coefficient according to the distance information on the distance between the imaging device and the object obtained via the distance information calculation unit (108). A conversion unit (111) sets a gradation conversion curve by using a histogram of a local region and the correction coefficient calculated by the correction coefficient calculation unit (112) and performs a gradation conversion process on the video signal.

Description

Video signal processing device, video signal processing method, and video signal processing program

Technical field

The present invention relates to a video signal processing device, a video signal processing method, and a video signal processing program for performing signal processing on a video signal, and in particular, a video signal processing device and a video signal processing for performing gradation conversion processing on a video signal. The present invention relates to a method and a video signal processing program.

Background technology

In general digital cameras and electronic cameras (video signal processing devices) such as video cameras, in order to prevent image quality degradation due to rounding errors such as gradation skip in digital signal processing, The gradation bit width may be set wider than the gradation bit width of the output signal. In this case, it is necessary to perform gradation conversion on the input signal so as to match the gradation bit width of the output system.

In the past, processing was performed by fixed gradation conversion for standard scenes. In addition, a method has been proposed in which the video signal is divided into a plurality of areas without performing fixed gradation conversion, and gradation conversion is performed independently for each area. For example, Japanese Patent Laid-Open No. 2 0 0 1 — 1 1 8 0 6 2 discloses an example in which a video signal is divided into a plurality of regions based on texture information, and gradation conversion is adaptively performed on each divided region. It is disclosed.

Japanese Patent Application Laid-Open No. 5-6820 205 discloses an example in which a main subject region is detected using distance information to a subject, and an exposure amount at the time of photographing is controlled with emphasis on the detected region. Has been. In the method disclosed in Patent Document 2, the main subject area is extracted based on the measured information, and the exposure amount is controlled with emphasis on the extracted area. Exposure. In the prior art, gradation compression is performed on the entire video signal on average, or fixed gradation conversion such as r conversion is performed. In this case, when the scene is different from a standard scene such as backlight, there is a problem that the main subject does not have an appropriate gradation and the image is not subjectively preferable.

In the method disclosed in Japanese Patent Laid-Open No. 2 0 0 1 — 1 1 8 0 6 2, tone conversion is performed independently for each region, so that a preferable image can be obtained even in a scene with a large contrast ratio such as backlight. There are no restrictions on individual tone conversion. For this reason, extreme gradation conversion may be performed, and side effects such as increased noise components, color reproduction failure, and image quality deterioration due to gradation skipping become new challenges. In addition, the same tone conversion process is performed when the regions existing in the foreground and the back of the image have the same histogram distribution, resulting in a flat image with no sense of contrast. There was a problem.

In the method disclosed in Japanese Patent Application Laid-Open No. 5-6 8 20 5, in a scene where the ratio of the main subject to the background is large, the background is not properly exposed, so that it is extremely bright. There is a problem of becoming darker or darker. In the above method, the closest area is the area of the main subject, but if there is another object in front of the object to be photographed as the main object, an appropriate exposure is applied to the object in front of it. Since there is a large amount of shooting, there is a problem that it is not possible to obtain a high-quality image with an appropriate gradation according to the main subject.

In view of the above problems, an object of the present invention is to provide a video signal processing device, a video signal processing method, and a video signal processing program for performing high-quality gradation conversion corresponding to a subject for a video signal. . Disclosure of invention

In order to achieve the above object, the video signal processing apparatus of the present invention provides an image pickup hand. In the signal processing apparatus, distance information acquisition means for acquiring distance information between the imaging object and the imaging means, and the distance. Gradation conversion means for performing gradation conversion processing on the target pixel of the video signal using information.

Embodiments relating to the present invention correspond to the first embodiment shown in FIGS. 1 to 7 and the second embodiment shown in FIGS. 8 to 13. The distance information acquisition means, which is the configuration of the present invention, corresponds to the distance information calculation unit 10 8 shown in FIGS. 1 and 8, and the gradation conversion means is shown in FIGS. 1, 2, 5, and 7. Applicable to the conversion unit 1 1 1. A preferred application example of the present invention is that the distance information calculation unit 10 8 shown in FIGS. 1 and 8 obtains the distance to the subject, and the conversion unit 1 1 1 provides the distance information for each pixel of interest. It is a video signal processing device that uses it to perform gradation conversion processing.

The present invention performs gradation conversion processing independently based on distance information for each target pixel of an input video signal. With such a configuration, a subjectively preferable video signal can be obtained even for a scene with a small contrast ratio.

In order to achieve the above object, the video signal processing method of the present invention includes a step of acquiring a video signal of an imaging target by an imaging unit, a step of acquiring distance information between the imaging target and the imaging unit, Performing a gradation conversion process on the target pixel of the video signal using the distance information.

(A) An application example of the video signal processing method of the present invention will be described together with its configuration. (A) corresponds to the first embodiment shown in FIGS. 1 to 7 and the second embodiment shown in FIGS. 8 to 13. The stage of acquiring the video signal to be imaged by the imaging means, which is the configuration of (A), corresponds to the imaging by the CCD 100 shown in FIGS. “The step of obtaining the distance information between the photographing object and the photographing means” is the distance information of FIG. 1 and FIG. The processing of the calculation unit 1 0 8 corresponds. The “gradation conversion process adaptively using the correction coefficient for the target pixel of the video signal using the distance information” is performed by the conversion unit 1 1 1 in FIG. 1 and FIG. Applicable. A preferred application example of (A) is the image processing method in the imaging device (video signal processing device) shown in FIGS. 1 and 8 as described above. However, the invention of (9) is not limited to the video signal processing method in the imaging apparatus having the configuration shown in FIGS. 1 and 8, and the video signal processing having a configuration for performing the processing of each stage described above. Any device is applicable.

(A) is the acquisition of the video signal to be imaged by the imaging means, the acquisition of the distance information between the imaging object and the imaging means, and the gradation conversion process for the target pixel of the video signal using the distance information. Since this process is performed in a series of steps, it is possible to generate high-quality images according to the subject with appropriate gradation and few side effects such as noise increase and color reproduction failure.

(B) Another video signal processing method of the present invention uses a step of acquiring a video signal of an imaging target by an imaging unit, a step of acquiring distance information between the imaging target and the imaging unit, and the distance information. A step of calculating a correction coefficient for the target pixel of the video signal, and a step of adaptively performing gradation conversion processing on the target pixel using the correction coefficient.

An application example of (B) will be described together with its configuration. (B) corresponds to the first embodiment shown in FIGS. 1 to 7 and the second embodiment shown in FIGS. 8 to 13. The stage of acquiring the image signal to be imaged by the imaging means, which is the configuration of (B), corresponds to the imaging by the CCD 100 shown in FIGS. The “step of acquiring distance information between the object to be imaged and the image capturing means” corresponds to the processing of the distance information calculating unit 108 in FIGS. “The step of calculating the correction coefficient for the target pixel of the video signal using the distance information” is performed by the processing of the correction coefficient calculation unit 1 1 2 in FIG. 1 and FIG. Applicable. “The stage of performing adaptive conversion and conversion processing on the target pixel using the correction coefficient” corresponds to the processing of the conversion unit 1 1 1 in FIGS.

A preferable application example of (B) is the image processing method in the imaging apparatus (video signal processing apparatus) shown in FIGS. 1 and 8 as described above. However, (B) is not limited to the video signal processing method in the imaging device having the configuration shown in FIGS. 1 and 8, and may be a video signal processing device having a configuration for performing the processing of each stage. If applicable.

(B) is the acquisition of the video signal to be imaged by the imaging means, the acquisition of distance information between the imaging object and the imaging means, the calculation of the correction coefficient for the target pixel of the video signal using the distance information, Since the process of adaptive gradation conversion using the correction coefficient for the pixel of interest is performed in a series of steps, side effects such as an increase in noise and failure of color reproduction at an appropriate gradation are performed. It is possible to generate a few high-quality images according to the subject.

(C) In order to achieve the above object, the video signal processing program according to the present invention includes a procedure for causing a computer to read an unprocessed video signal at the time of shooting, and distance information between the shooting target and the shooting means. And a procedure for performing a gradation conversion process on a target pixel of the video signal using the distance information.

An application example of (C) will be described together with its configuration. (C) corresponds to the first embodiment shown in FIGS. 1 to 7 and the second embodiment shown in FIGS. 8 to 13. (C) The “Procedure for reading an unprocessed video signal at the time of shooting” is to have a computer output the signal from the CCD 1 0 2 shown in FIG. This corresponds to the process of reading the video signal output from the control unit 1 15 as the header information. “Procedure for obtaining distance information between photographing object and photographing means” corresponds to the processing of the distance information calculating unit 108 shown in FIGS. 1 and 8 by the computer. "Using the distance information, the video signal The procedure for performing the gradation conversion processing on the target pixel corresponds to the processing of the conversion unit 1 1 1 in FIGS. 1 and 8 by the computer.

(D) Another video signal processing program of the present invention includes a procedure for causing a computer to read an unprocessed video signal at the time of shooting, a procedure for calculating distance information from the video signal that has been subjected to signal processing, and the distance A procedure for calculating a correction coefficient based on the information; a procedure for sequentially extracting a local region centered on a pixel of interest from the signal-processed video signal; a procedure for creating a histogram of the extracted region; and the correction A procedure for performing clipping processing on the histogram based on a coefficient, a procedure for generating a gradation conversion curve by accumulating and normalizing the histogram after the clipping processing, A procedure for performing gradation conversion processing on a pixel of interest based on a gradation conversion curve;

Is executed.

An application example of (D) will be described together with its configuration. (D) corresponds to the first embodiment shown in FIG. 6 and the second embodiment shown in FIG. The configuration of (D) corresponds to each Step in Fig. 6 and Fig. 13. In other words, “Procedure to read unprocessed video signal at the time of shooting” is “Ste P 1”, “Procedure to calculate distance information from signal processed video signal” is “Step 5,” The procedure for calculating the correction coefficient based on the distance information is “Step 6,” and the “procedure for sequentially extracting a local region centered on the pixel of interest from the signal-processed video signal” is the Step 3, The procedure for creating a histogram of the extracted area ”is Step 4,“ The procedure for performing the clipping process on the histogram based on the correction coefficient ”is Step 7, “Procedure for generating gradation conversion curve by accumulating and normalizing histogram after post-processing” Step 8, “Procedure for performing gradation conversion processing for target pixel based on above gradation conversion curve” "Corresponds to Step 9, respectively. According to the video signal processing program of the present invention, it is possible to obtain a high-quality image corresponding to a subject with few side effects such as an increase in noise and a failure in color reproduction.

According to the present invention, it is possible to obtain a video signal processing device, a video signal processing method, and a video signal processing program that perform high-quality gradation conversion according to a subject with few side effects on a video signal. Brief Description of Drawings

FIG. 1 is a block diagram of the first embodiment.

Figure 2 is an illustration of the clip process.

Fig. 3 is an explanatory diagram of linear interpolation.

FIG. 4 is a second configuration diagram of the conversion unit 11 1 1 in the first embodiment. FIG. 5 is a configuration diagram of the conversion unit 1 1 1 in the first embodiment.

FIG. 6 is a flowchart of the gradation conversion process in the first embodiment.

FIG. 7 is a third block diagram of the converter 1 1 1.

FIG. 8 is a configuration diagram in the second embodiment.

FIG. 9 is a block diagram of the shooting situation estimation unit 1 1 7.

FIG. 10 is an explanatory diagram of a division pattern for evaluation photometry.

Figure 11 is an explanatory diagram of shooting scene classification patterns.

Fig. 12 is a characteristic diagram of gradation curve setting for classification patterns.

FIG. 13 is a flowchart of the gradation conversion process in the second embodiment. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. The first embodiment is shown in FIGS. Figure 1 shows the first embodiment 2 is a block diagram of the conversion unit 1 1 1, FIG. 3 is an illustration of clip processing, FIG. 4 is an illustration of linear interpolation, and FIG. 5 is a second configuration of the conversion unit 1 1 1. FIG. 6 is a flowchart of the tone conversion processing in the first embodiment, and FIG. 7 is a third configuration diagram of the conversion unit 11 1 1.

The configuration of the first embodiment will be described with reference to FIG. The image taken through the lens system 100, the aperture 1 0 1 and the CCD 1 02 is A / D converter 1 04 (hereinafter referred to simply as AZD 1 04 in the description and drawings of the present invention). Is converted into a digital signal. The video signal from AZD 104 is transferred to the photometry evaluation unit 106, the distance information calculation unit 108, and the signal processing unit 110 via the buffer 105. The photometric evaluation unit 106 is connected to the aperture 100 1 and the C CD 102, and the distance information calculation unit 108 is connected to the lens control unit 107 and the correction coefficient calculation unit 1 1 2. The lens control unit 107 is connected to the AF motor 103, and the infrared sensor 109 is connected to the distance information calculation unit 108. The correction coefficient calculation unit 1 1 2 is connected to the conversion unit 1 1 1, and the signal processing unit 1 1 0 is connected to the conversion unit 1 1 1. The conversion unit 1 1 1 is connected to the compression unit 1 1 3, and the compression unit 1 1 3 is connected to the output unit 1 1 4. Microcomputer control unit 1 1 5 is AZD 104, photometry evaluation unit 1 06, lens control unit 1 07, distance information calculation unit 1 08, infrared sensor 1 09, signal processing unit 1 1 0, conversion unit 1 1 1, Correction coefficient calculation unit 1 1 2, Compression unit 1 1 3, Output unit 1 14 In addition, an external 1 ZF unit 1 1 6 with a power switch, shut-down turbotan, and an interface for switching various modes during shooting is also connected to the control unit 1 1 5 in both directions. Yes.

Next, the first embodiment will be described along the signal flow shown in FIG. Set the shooting parameters such as ISO sensitivity and shutter speed via the external part 1 1 1 6 and then press the shutter button halfway. 60758

/ Pre-shooting mode can be entered by selecting this option. The video signal photographed through the lens system 100, the aperture 100 1, and the CCD 102 is converted into a digital signal by the AZD 104 and transferred to the buffer 105. In this embodiment, the gradation width of the digitized video signal is, for example, 12 bits. The video signal in the buffer 105 is transferred to the photometric evaluation unit 106, the distance information calculation unit 108, and the signal processing unit 110. The photometric evaluation unit 106 obtains the luminance level in the video signal, controls the aperture 1010 and the electronic shutter speed of the CCD 102, etc. so as to achieve proper exposure. The distance information calculation unit 108 uses the external infrared sensor 110 to calculate distance information indicating the distance between the imaging device and the subject. Alternatively, the contrast information of the video signal is detected, and the in-focus image is obtained by controlling the AF motor 103 so that the contrast is maximized at the position where you want to obtain distance information in the image. To do. Distance information is calculated from the lens position (or AF motor control information) when the focused image is obtained. The calculated distance information is transferred to the lens control unit 107 and the correction coefficient calculation unit 1 1 2.

Next, full shooting is performed by fully pressing the shutter button via the external 1 F section 1 1 6, and the video signal is transferred to the buffer 105 in the same way as pre-imaging. Tree shooting is performed based on the exposure conditions obtained by the photometry evaluation unit 106 and the focusing conditions obtained by the lens control unit 107, and these shooting conditions are transferred to the control unit 1 15 Is done. The video signal in the buffer 105 is transferred to the signal processing unit 110. The signal processing unit 1 1 0 reads a single-plate video signal on the buffer 1 05 based on the control of the control unit 1 15 and performs a known interpolation process, white balance process, enhancement process, etc. Generate a status signal and transfer it to the converter 1 1 1. In the correction coefficient calculation unit 1 1 2, the gradation is calculated based on the distance information indicating the distance between the imaging device and the subject obtained through the distance information calculation unit 1 08. A correction coefficient for changing the physical properties of the conversion curve is calculated. For example, the correction coefficient is calculated so that the gradient of the gradation conversion curve is monotonously suppressed with respect to the distance. In the present embodiment, it is assumed that the distance information obtained in the distance information calculation 108 is a plurality of points in the video signal. The distance information at positions corresponding to multiple areas on the image is calculated using an external infrared sensor 1 0 9 or the contrast of the video signal at positions corresponding to multiple areas including pixels or multiple pixels The distance information of multiple points is obtained by calculating the distance information from the target. The conversion unit 1 1 1 sets a gradation conversion curve using the local area histogram and the correction coefficient calculated by the correction coefficient calculation unit 1 1 2, and performs gradation conversion processing on the video signal. Do. The video signal after the gradation conversion process is transferred to the compression unit 1 1 3. In the compression unit 1 1 3, the video signal obtained through the conversion unit 1 1 1 is subjected to a compression process such as known JPEG, and the compressed video signal is transferred to the output unit 1 1 4 . The output unit 1 1 4 records and stores the compressed signal in a memory card.

FIG. 2 shows an example of the configuration of the conversion unit 1 1 1. In FIG. 2, the conversion unit 1 1 1 includes a buffer 2 0 0, a local region extraction unit 2 0 1, a representative point extraction unit 2 0 2, a histogram creation unit 2 0 3, and a clipping unit 2 0 4. It consists of an accumulation normalization unit 205, a gradation curve creation unit 206, and a gradation conversion unit 2007. The signal processing unit 10 8 is connected to a nother 2 0 0, and the nofer 2 0 0 and the representative point extracting unit 2 0 2 are connected to the local region extracting unit 2 0 1. The local region extraction unit 20 1 is connected to the histogram creation unit 20 3, and the histogram creation unit 2 0 3 and the correction coefficient calculation unit 1 1 2 are connected to the clipping unit 2 0 4. ing.

The clipping unit 20 4 is connected to the cumulative normalization unit 2 05, and the cumulative normalization unit 2 0 5 is connected to the gradation curve creation unit 2 6. The tone curve generator 2 0 6 is connected to the tone converter 2 0 7 The unit 2 0 7 is connected to the compression unit 1 1 3. The control unit 1 1 5 includes a local region extraction unit 2 0 1, a representative point extraction unit 2 0 2, a histogram creation unit 2 0 '3, a clipping unit 2 0 4, and a cumulative normalization unit 2 0 It is connected to the 5 gradation curve creation unit 206 and the gradation conversion unit 206 in both directions.

The video signal transferred from the signal processing unit 110 is stored in the buffer 200. The representative point extraction unit 20 2 extracts one or more representative points of the video signal from which the distance information calculation unit 1 0 8 has obtained distance information to the subject via the control unit 1 15. . The local region extraction unit 20 1 is a rectangular region having a predetermined size centered on the pixel of the representative point extracted by the representative point extraction unit 202. For example, in this example, the local region extraction unit 20 1 is a local region of 16 × 16 pixels. Extract region. The histogram creation unit 20 03 creates a histogram for each local region extracted by the local region extraction unit 20 1, and transfers the created histogram information to the clipping unit 204. The clipping unit 20 4 is created by the histogram creation unit 2 0 3 based on the correction coefficient obtained by the correction coefficient calculation unit 1 1 2 using the distance information of the representative point to be processed. Clip the histogram.

Figure 3 is an explanatory diagram of the clip process. Fig. 3 (a) shows the original histogram from the histogram creation unit 203 and the clip value for the frequency on the vertical axis. Fig. 3 (b) shows a histogram in which the clip value is replaced with the clip frequency more than the clip value by clip processing. Fig. 3 (c) shows the tone conversion curve obtained by accumulating and normalizing the original histogram and the histogram after clipping. The clipping process suppresses the histogram frequency, and as a result, the gradient of the tone conversion curve is suppressed. Setting the clip value lower in the clip process lowers the signal gain and does not significantly change the contrast in the local region. On the other hand, if the clip value in the clip process is set high, the gain for the signal is increased, This has the effect of increasing the contrast in the local region. In this embodiment, the clip C is calculated by equation (1).

[Equation 1]

C = vk (d) 'N (1) where W is a predetermined weighting factor and N is the total number of pixels in the local region. k (d) represents the correction coefficient for the distance d obtained from the correction coefficient calculation unit 1 1 2. The correction coefficient k (d) is preferably monotonically decreasing with respect to the distance d indicated by the distance information, and is represented by, for example, a function of equation (2).

[Equation 2]

(2) k (d) = const I d where c o n st represents a predetermined constant. As shown in equation (2), the closer the distance from the imaging device to the subject, the higher the clip value for generating the gradation conversion curve, and the contrast conversion is performed by the gradation conversion process. Make sure that the height is high. The clipped histogram is transferred to the accumulation normalization unit 205. The cumulative normalization unit 205 creates a cumulative histogram by accumulating the histogram, and generates a gradation conversion curve by normalizing it according to the gradation width.

In the present embodiment, since the gradation width of the video signal is 12 bits, the gradation conversion curve is a 12-bit input and a 12-bit output. The gradation conversion curve is transferred to the gradation curve creation unit 206. The gradation curve creation unit 2,06 calculates a gradation conversion curve for all the pixels of the video signal based on the gradation conversion curves at a plurality of points obtained by the cumulative normalization unit 205. When there is a pixel position for which distance information is not available, the gradation conversion curve is calculated by, for example, the linear interpolation process shown in FIG. In this embodiment, the gradation conversion curve f (x, y) of the target pixel (x, y) is used as the surrounding four pixels. Calculate with equation (3).

[Equation 3]

+ (Μ + ΐ-χ) ([ν]-ΧΛΜ, Μ + ΐ)

) + (Μ- (]) 7 (Μ + ΐ, Μ + ι) ( ₃ )

In the calculation of Eq. (3), the values of points A and B in Fig. 4 are first obtained by linear interpolation using the left and right values of each, and the obtained points A and B in the vertical direction. Perform linear interpolation. Similarly, the values of points C and D in the figure are obtained by linear interpolation using the upper and lower values, respectively, and then linearly interpolated in the horizontal direction of points C and D to obtain the pixel (x, y ) Gradation conversion curve f (x, y). The calculated gradation conversion curve is transferred to the gradation conversion unit 2 07. The gradation conversion unit 20 7 performs gradation conversion processing on the pixel of interest on the buffer 2 0 0 based on the gradation conversion curve transferred from the gradation curve creation unit 2 06. After that, division processing is performed so as to conform to the gradation width at the time of output (assuming 8 bits in this embodiment). The 8-bit video signal is transferred to the compression unit 1 1 3. In the above example, the gradation conversion curve based on the local area histogram is calculated, but it is not necessary to be limited to such a configuration.

For example, as shown in FIG. 5, a configuration using a preset gradation conversion curve is also possible. FIG. 5 shows the configuration of the conversion unit 1 1 1 shown in FIG. 2, in which a local region extraction unit 2 0 1, a histogram creation unit 2 0 3, a clipping unit 2 0 4, and a cumulative normalization unit 2 0 5 is omitted, and a gradation conversion curve ROM 2 08 and a gradation curve changing unit 2 9 are added. The basic configuration is the same as that of the converter shown in Fig. 2, and the same configuration is given the same name and number. Only the differences will be described below.

The tone conversion curve ROM 2 08 is connected to the tone curve changing unit 2 09. The correction coefficient calculation unit 1 1 2 is connected to the gradation curve changing unit 2 09 and the gradation curve changing unit 2 0 9 is connected to the gradation curve creating unit 2 0 6 The The gradation curve changing unit 2109 corrects the gradation conversion curve read from the gradation conversion curve ROM 2 0 8 based on the correction coefficient of the correction coefficient calculation unit 1 1 2. For example, if the tone conversion curve for the input luminance value i (i = 0 to 1) is given by Ύ (i), the corrected tone conversion curve '(i) is expressed by equation (4). The

[Expression 4] f (0 = 7 (i) ^Kd) (4) Gradation Conversion Curve The relationship between the luminance value and the gradation conversion curve is recorded in advance in ROM 2 ⁰⁸ . Here, for example, the correction coefficient k (d) for the distance d is desirably set so that the value of the correction coefficient k (d) increases in accordance with the magnitude of the distance d. In this way, the processing speed can be increased by setting a standard gradation conversion curve in advance. The corrected gradation conversion curve is transferred to the gradation conversion creation unit 206. The subsequent processing is equivalent to the processing described in FIG.

With the configuration shown in FIG. 5, an adaptive tone conversion curve corresponding to the distance to the subject can be obtained. With such an adaptive tone conversion curve, a subjectively pleasing image with a sense of depth can be obtained with appropriate tone characteristics according to the distance. The gradation conversion processing has the same configuration as the conventional one, and the correction processing is added. Therefore, the compatibility with the conventional device is high and mounting is easy. Since the gradation conversion curve is set based on the histogram, it is possible to adaptively perform gradation conversion processing for various scenes, and high-quality video signals even for scenes with large contrast ratios. Can be obtained. In addition, by setting restrictions on gradation characteristics independently for each region, side effects such as an increase in noise components and failure of color reproduction can be suppressed. Furthermore, high-speed processing is possible by setting a standard tone conversion curve fixedly. In the above configuration example, processing by hardware is assumed, but it is not necessary to be limited to such a configuration. The video signal processing can be configured through a predetermined stage, that is, a video signal processing method. In addition, the signal from CCD 102 can be output as raw data as raw data, and the shooting information from control unit 1 15 can be output as header information, which can be processed separately by software. It is. FIG. 6 is a flowchart relating to the software processing of the gradation conversion processing in the first embodiment according to the present invention. In Ste 1, read header information including unprocessed video signals and supplementary information such as imaging conditions. Performs known interpolation processing, white balance processing, enhancement processing, etc. at Ste P2. On the other hand, the distance information of the video signal processed in Step 2 is calculated in Step 5, and the correction coefficient is calculated in Step 6 based on the distance information. In addition, the video signal processed in Step 2 sequentially extracts local regions centered on the representative point in Step 3 on the other side. Next, in Step 4, create a histogram of the extracted local region.

In Step 7, the clipping process is performed on the histogram obtained in Step 4 based on the correction factor from Step 6. At St ρ 8, the gradation conversion curve is generated by accumulating and normalizing the histogram after clipping. Steps 3 to 8 are performed on all representative points for which distance information can be calculated. In Step 9, gradation conversion processing is performed on the pixel of interest based on the gradation conversion curve from Step 8. In Step 1 0, it is determined whether or not all pixels of interest have been completed. If all pixels of interest have not been completed, the process proceeds to Step 9. If all pixels of interest have been completed, St 1 is used. Move to 1. At St e p 1 1, compression processing such as known JPEG is performed. At Step 1 2, the processed signal is output and the program ends.

In the configuration example of FIG. 5 above, each image is based on the distance information to the subject. Although the correction coefficient for the element is calculated and the gradation conversion curve is set using the calculated correction coefficient, the present invention is not limited to this. A configuration can also be used in which the tone conversion curve is directly set using distance information without using a correction coefficient. For example, a gradation conversion curve corresponding to the distance information value is set in advance, a gradation conversion curve corresponding to the distance information is selected for each pixel, and gradation conversion processing is performed using the selected gradation conversion curve. A way to do this is conceivable.

FIG. 7 shows an example of the configuration of the conversion unit 1 1 1 for performing the above processing. In the configuration of FIG. 7, the correction coefficient calculation unit 1 1 2 and the gradation curve changing unit 2 09 are omitted and the gradation curve setting unit 2 10 is added to the configuration shown in FIG. The basic configuration is equivalent to the conversion unit 1 1 1 shown in Fig. 5, and the same configuration is given the same name and number. Only the differences from the configuration shown in FIG. 5 will be described below.

In FIG. 7, the tone conversion curve R O M 2 0 8 is connected to the tone curve setting unit 2 1 0. The tone curve setting unit 2 1 0 is connected to the tone curve creation unit 2 0 6, and the control unit 1 1 5 is connected to the tone curve setting unit 2 1 0. In the gradation conversion curve R O M 208, the relationship between the distance information and the gradation conversion curve is recorded in advance. The gradation curve setting unit 2 10 reads a gradation conversion curve corresponding to the distance information from the control unit 1 15 from the gradation conversion curve R O M 208.

The subsequent processing is the same as in Figure 5. Note that the present invention is not limited to the configuration in which the tone conversion curve R O M 208 is stored with the tone conversion curve corresponding to the distance information as in this example. For example, in the gradation conversion curve ROM 208, the relationship between the distance information and the output signal value after gradation conversion corresponding to the luminance information is stored in advance as a table, and the distance information and luminance of the target pixel are stored. It is also possible to perform gradation conversion processing with reference to the table based on the information.

8 to 13 show a second embodiment of the present invention. FIG. 8 is a configuration diagram of the second embodiment, FIG. 9 is a configuration diagram of the shooting situation estimation unit 1 1 7, and FIG. Is an explanatory diagram of the division pattern for evaluation metering, Fig. 11 is an explanatory diagram of the classification pattern of the shooting scene, Fig. 12 is a characteristic diagram of gradation curve setting for the classification pattern, and Fig. 13 is in the second embodiment. This is a flowchart of gradation conversion processing.

The configuration of the second embodiment of the present invention will be described with reference to FIG. This embodiment has a configuration in which a shooting estimation situation section 1 17 is added to the first embodiment. The basic configuration is the same as in the first embodiment, and the same configuration is given the same name and number. Only the portions different from the first embodiment will be described below. The photometric evaluation unit 1 06 and the lens control unit 1 0 7 are connected to the imaging state estimation unit 1 1 7. The shooting state estimation unit 1 1 7 is connected to the correction coefficient calculation unit 1 1 2, and the control unit 1 1 5 is connected to the shooting state estimation unit 1 1 7 in both directions.

Next, a second embodiment will be described along the signal flow shown in FIG. The second embodiment is basically the same as the first embodiment, and only different parts will be described. The video signal in the three-plate state that has been subjected to known interpolation processing, white balance processing, enhancement processing, and the like by the signal processing unit 110 is transferred to the conversion unit 110. Based on the exposure conditions obtained by the photometric evaluation section 10 06 and the focusing conditions obtained by the lens control section 107, the shooting condition estimation section 1 1 7 is used for landscapes, portraits, backlighting, etc. The shooting situation is estimated, and the estimated shooting situation is transferred to the correction coefficient calculation unit 1 1 2.

The correction coefficient calculation unit 1 1 2 calculates the correction coefficient based on the distance information to the subject obtained by the distance information calculation unit 10 8 and the information on the shooting situation from the shooting situation estimation unit 1 1 7. To do. The conversion unit 1 1 1 sets a gradation conversion curve using the local area histogram and the correction coefficient calculated by the correction coefficient calculation unit 1 1 2, and performs gradation conversion processing on the video signal. I do. The video signal after the gradation conversion processing is transferred to the compression unit 1 1 3.

Fig. 9 shows an example of the configuration of the shooting situation estimation unit 1 1 7. The situation estimation unit 1 1 7 includes a subject distribution estimation unit 3 0 0, a focus position estimation unit 3 0 1, and an integration unit 3 0 2. Photometric evaluation section 1 0 6 and lens control section 1

Information from 07 is transferred to the subject distribution estimation unit 3 0 0 and the in-focus position estimation unit 3 0 1. The subject distribution estimation unit 3 0 0 and the in-focus position estimation unit 3 0 1 are connected to the integration unit 3 0 2, and the integration unit 3 0 2 is connected to the correction coefficient calculation unit 1 1 2. The in-focus position estimation unit 3 0 1 obtains distance information from the distance information calculation unit 1 0 8 via the control unit 1 1 5. Based on the distance information at the in-focus position, the subject is classified into two types, for example, landscape (5 m or more) and portrait (1 m to 5 m). 0 Transfer to 2.

On the other hand, the subject distribution estimation unit 3 0 0 obtains information on photometric evaluation via the control unit 1 1 5. FIG. 10 is an explanatory diagram showing an example of a division pattern for photometric evaluation. 13 Divided into three areas, and the luminance value (a

1 = 1 to 1 3) The subject distribution estimation unit 30 1 calculates, for example, a parameter of equation (5) as photometric information from the luminance value a i of each region. Thereby, the luminance distribution is calculated.

[Equation 5]

S = ai-a (o

S is the absolute value of the difference between the center area and the edge area. For example, when shooting with a portrait, the value of S is a large positive value when shooting in the dark. When shooting in a backlight environment, the S value becomes a large negative value. The subject distribution estimation unit 300 calculates the photometry information as described above and transfers it to the integration unit 302. The integration unit 30 2 estimates the shooting situation based on the focusing and photometric information.

FIG. 11 is an explanatory diagram showing a case where four types of shooting situations are estimated from focusing and photometry information. In the example in Figure 11 1, there are three different port rates (T It is classified into ypel ~ Ty pe 3) and landscape (T ype 4). When it is classified as a portrait from the focus information, the standard scene (T ype 1) and the dark place strobo scene (Ty pe) are compared by comparing the parameter S of the photometry information with a predetermined threshold th. 2) and backlit scenes (Type 3). The combining unit 302 transfers the above photographing state to the correction coefficient calculating unit 1 1 2.

FIG. 12 is an explanatory diagram showing a correction coefficient calculation method in the correction coefficient calculation unit 1 1 2 corresponding to the four types of shooting situations shown in FIG. For example, when classified into Type 1, the gain is calculated so that the gain is greatly increased for pixels closer to the subject and the gain is not changed substantially for pixels farther away. Specifically, the clip value corresponding to the distance is set from equation (2), and the histogram generated from the region near the representative point pixel is clipped, or the optimum value according to the distance. Select the tone conversion curve to be processed. When classified into Type 2, the brightness of pixels that are close to each other increases overall, and the brightness of pixels that are farther away decreases overall. The correction coefficient is calculated so that the gain increases as the pixel increases. Specifically, the clip value is set by adding the distance d to the constant const in Eq. (2), and the clip processing of the histogram is performed, or the optimum according to the distance. Select the tone conversion curve.

When the shooting situation is classified as Type 3, the closer the pixels are, the lower the brightness overall, whereas the farther away the pixels are, the higher the brightness is, so the closer the pixels are, the closer the distance is. Select a correction factor or gradation conversion curve so that the gain is increased and the gain is decreased as the pixel is further away. In Type 4, since the main subject is located far away, the correction coefficient or gradation conversion curve is selected so that the farther away the pixels, the higher the gain. With the above configuration, it is possible to obtain an adaptive tone conversion curve based on information on the distance to the subject and the shooting situation. Since the shooting scene is estimated at the time of shooting and the correction coefficient is obtained, it is possible to control gradation conversion processing for each shooting scene, and a subjectively favorable image corresponding to the subject can be obtained. In the above configuration example, processing by hardware is assumed, but it is not necessary to be limited to such a configuration. It can be configured as a configuration in which video signal processing is performed through a predetermined stage, that is, a video signal processing method. In addition, the signal from the CCD 10 2 is processed as raw data, and the shooting information from the control unit 1 15 is output as header information. A configuration in which processing is performed by a key is also possible.

FIG. 13 shows a flowchart relating to the software processing of the gradation conversion processing according to the second embodiment of the present invention. Note that the same step numbers are assigned to the same processing steps as the flowchart of the gradation conversion processing in the first embodiment shown in FIG. At Step 1, the header information including the unprocessed video signal and accompanying information such as imaging conditions is read. At Step 2, perform known interpolation processing, white balance processing, and enhancement processing. On the other hand, the video signal that has undergone signal processing in Step 2 calculates distance information in Step 5 and obtains focusing information as described above based on the distance information. For example, the focus position is estimated for two types of landscapes and portraits. Based on the photometric evaluation, calculate the luminance distribution that changes according to the strobe, for example, in backlight or in dark places. In Ste P 14, the above focusing position and shooting environment are integrated, and for example, the shooting situation is classified into four types shown in FIG. Step 6 calculates the correction factor based on the estimated scene type.

On the other hand, the local signal centered on the target pixel is sequentially extracted from the video signal subjected to signal processing in Step 2 on the other hand. Next, in Step 4, create a histogram of the extracted area. To S tep 7 Then, based on the correction coefficient from Step 6, the clipping process is performed on the t-gram obtained in Step 4. In Step 8, a gradation conversion curve is generated by accumulating and normalizing the histogram after clipping. The processing so far is performed for all representative points for which distance information can be calculated. In Step 9, gradation conversion processing is performed on the target pixel based on the gradation conversion curve from Step 8. In Step 1 0, it is determined whether or not all pixels of interest have been completed. If all pixels of interest have not been completed, the process proceeds to Step 3. If all pixels of interest have been completed, Step 1 1 is performed. Transition. At Stepll, compression processing such as well-known JPEG is performed. At Step 1 2, the processed signal is output and the program ends. In the above configuration example, the shooting situation is automatically estimated based on the focusing and photometry information, but the present invention is not limited to this, and a configuration in which the shooting situation is manually specified is also possible. Industrial applicability

As described above, according to the present invention, it is possible to provide a video signal processing device, a video signal processing method, and a video signal processing program that perform high-quality gradation conversion according to a subject. Such a video signal processing apparatus is particularly preferably applied to an imaging apparatus.

Claims

Scope of request S ''

1. In a video signal processing apparatus that performs gradation conversion processing on a video signal to be captured obtained from an imaging means,

Distance information acquisition means for acquiring distance information between the imaging object and the imaging means; and gradation conversion means for performing gradation conversion processing on a target pixel of the video signal using the distance information. A video signal processing device.

2. The gradation converting means includes a histogram calculating means for calculating a histogram of the target pixel and a neighboring area, and a clipping means for performing a clipping process on the histogram based on the distance information. The video signal processing apparatus according to claim 1, further comprising a gradation conversion curve setting unit that sets a gradation conversion curve based on a histogram after the clip processing.

3. The video signal processing apparatus according to claim 1, wherein the gradation conversion means further includes gradation conversion curve storage means for storing a predetermined gradation conversion curve for the distance information.

4. Correction coefficient calculation means for calculating a correction coefficient for the date-added pixel of the video signal using the distance information, and gradation conversion means for performing gradation conversion processing on the target pixel using the correction coefficient. The video signal processing apparatus according to claim 1, further comprising:

5. Estimating the shooting situation based on the luminance distribution calculating means for calculating the luminance distribution of the video signal, the distance information acquired by the distance information acquiring means, and the luminance distribution calculated by the luminance distribution calculating means. A shooting condition estimating means; a correction coefficient calculating means for calculating a correction coefficient for an injection date pixel of the video signal using the distance information and the shooting situation information; and a gradation using the correction coefficient for the target pixel. The video signal processing apparatus according to claim 1, further comprising: gradation conversion means for performing conversion processing.

6. The gradation converting means includes a histogram calculating means for calculating a histogram of the date-of-date pixel and a neighboring area, and a gradation for setting a gradation conversion curve based on the correction coefficient and the histogram. 5. The video signal processing apparatus according to claim 4, further comprising conversion curve setting means.

7. The gradation converting means includes a histogram calculating means for calculating a histogram of the pixel of interest and a neighboring area, and a clipping process for performing a tipping process on the histogram based on the correction coefficient. 5. The video signal processing apparatus according to claim 4, further comprising: means; and gradation conversion curve setting means for setting a gradation conversion curve based on the histogram after the clip processing.

8. The gradation conversion means includes gradation conversion curve storage means for storing a predetermined gradation conversion curve for luminance, and gradation curve changing means for changing the gradation conversion curve based on the correction coefficient. 5. The video signal processing apparatus according to claim 4, further comprising:

9. Acquiring a video signal to be imaged by the imaging means; acquiring distance information between the imaging object and the imaging means; and using the distance information for a target pixel of the video signal. A video signal processing method comprising: a step of performing a tone conversion process.

1 0. Acquiring a video signal to be imaged by the imaging means, acquiring distance information between the imaging object and the imaging means, and a correction coefficient for the target pixel of the video signal using the distance information. And a step of adaptively performing gradation conversion processing on the pixel of interest using the correction coefficient. A video signal processing method comprising:

1 1. A procedure for causing a computer to read an unprocessed video signal at the time of shooting, a procedure for acquiring distance information between a shooting target and a shooting means, and a target pixel of the video signal using the distance information. A procedure for performing gradation conversion processing and a video signal processing program characterized by executing Mu.

1. A procedure for causing a computer to read an unprocessed video signal at the time of shooting; a procedure for calculating distance information from the video signal that has been signal-processed; a procedure for calculating a correction coefficient based on the distance information; A procedure for sequentially extracting a local region centered on a pixel of interest from the signal-processed video signal, a procedure for creating a histogram of the extracted region, and a histogram for the histogram based on the correction coefficient. A procedure for performing the ripping process, a procedure for generating a gradation conversion curve by accumulating and normalizing the histogram after the clipping process, and a step for the target pixel based on the gradation conversion curve. A video signal processing program characterized by executing a key conversion process and