JP2009253834A - Image signal processing unit, image capturing system, image signal processing program, and image signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image signal processing unit, or the like capable of more appropriately performing gradation conversion processing to an image signal according to the dynamic range of an object. <P>SOLUTION: The image signal processing unit has: an AE sensor 105 for measuring the luminance of an object to acquire the size of the luminance range; a gradation conversion characteristic calculation section 111 for calculating gradation conversion characteristics, based on the pixel value of a pixel included in the image signal of an image, where the object of which luminance has been measured is photographed; a correction section 112 for correcting the gradation conversion characteristics so that the gradation conversion characteristics become closer to those calculated by the gradation conversion characteristic calculation section 111 when the area of the luminance range acquired by the AE sensor 105 is relatively small than when the luminance range acquired by the AE sensor 105 is relatively large; and a gradation conversion section 113 for performing the gradation conversion of an image signal by the corrected gradation conversion characteristics. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image signal processing apparatus, an imaging system, an image signal processing program, and an image signal processing method that can perform tone conversion on an image signal.

  In current digital still cameras, video cameras, and the like, gradation conversion processing is performed on acquired video signals in consideration of display system characteristics and human visual characteristics.

  Further, in determining the gradation conversion characteristics to be applied to the image signal, a technique for correcting the basic gradation conversion characteristics using information on the image signal itself has been developed.

  For example, Japanese Unexamined Patent Application Publication No. 2004-282282 describes a technique for correcting a dynamic range compression curve based on image data obtained from an image sensor.

  Japanese Patent Application Laid-Open No. 2003-179809 describes a technique for determining whether or not the luminance distribution needs to be corrected based on the luminance distribution state of the image obtained from the solid-state imaging device.

Then, by performing the gradation conversion processing as described above on the image signal, it is possible to improve whiteout and blackout in the image to some extent.
JP 2004-282282 A JP 2003-179809 A

  However, when the dynamic range of the subject is significantly different from the dynamic range of the imaging device, it is difficult to sufficiently improve the image quality even if gradation conversion processing is performed, and in some cases, the image quality may be deteriorated. It can be.

  Even when the same gradation conversion processing is performed on an image signal in which a main subject such as a person exists, the image quality improvement effect may differ depending on the state of the main subject.

  The present invention has been made in view of the above circumstances, and an image signal processing device, an imaging system, an image signal processing program, which can perform gradation conversion processing on an image signal more appropriately according to the dynamic range of a subject, An object of the present invention is to provide an image signal processing method.

  In order to achieve the above object, an image signal processing apparatus according to a first aspect of the present invention is an image signal processing apparatus that can perform gradation conversion of an image signal, and measures the luminance of the subject to measure the luminance of the subject. A luminance range acquisition unit that acquires a width of a luminance range; a gradation conversion characteristic calculation unit that calculates a gradation conversion characteristic based on a pixel value of a pixel included in an image signal of an image in which the subject is captured; and The gradation calculated by the gradation conversion characteristic calculation unit based on the luminance range acquired by the luminance range acquisition unit when the luminance range is relatively small compared to when the luminance range is relatively large A correction unit that corrects the gradation conversion characteristic so as to approach the conversion characteristic; and a gradation conversion unit that performs gradation conversion of the image signal using the gradation conversion characteristic corrected by the correction unit. It is.

  The imaging system according to the second aspect of the present invention is an imaging system that can perform gradation conversion of an image signal, and an imaging unit that images a subject and outputs the image signal, and a subject that is imaged by the imaging unit A luminance range acquisition unit that measures the luminance of the subject and acquires the width of the luminance range of the subject, and calculates a gradation conversion characteristic based on a pixel value of a pixel included in an image signal of an image in which the subject is captured Based on the width of the luminance range acquired by the gradation conversion characteristic calculation unit and the luminance range acquisition unit, the gradation conversion is larger when the luminance range is relatively smaller than when the luminance range is relatively large. A correction unit for correcting the gradation conversion characteristic so as to approach the gradation conversion characteristic calculated by the characteristic calculation unit, and a level for gradation conversion of the image signal using the gradation conversion characteristic corrected by the correction unit. A tone conversion unit, That.

  Further, an image signal processing program according to the third aspect of the present invention is an image signal processing program for causing a computer to perform gradation conversion of an image signal, and based on the pixel value of a pixel included in the image signal. A gradation conversion characteristic calculating step for calculating a gradation conversion characteristic and a relatively large brightness range based on the brightness range of the subject acquired at the time of imaging A correction step for correcting the gradation conversion characteristic so as to approach the gradation conversion characteristic calculated by the gradation conversion characteristic calculation step, and the image using the gradation conversion characteristic corrected by the correction step. And a gradation conversion step for gradation conversion of a signal.

  The image signal processing method according to the fourth aspect of the present invention is an image signal processing method for gradation-converting an image signal, wherein gradation conversion characteristics are based on pixel values of pixels included in the image signal. And a gradation conversion characteristic calculating step for calculating the brightness range, and when the brightness range is relatively small based on the brightness range of the subject acquired at the time of imaging, A correction step for correcting the gradation conversion characteristic so as to approach the gradation conversion characteristic calculated by the tone conversion characteristic calculation step; and the gradation conversion of the image signal using the gradation conversion characteristic corrected by the correction step. And a tone conversion step.

  According to the image signal processing device, the imaging system, the image signal processing program, and the image signal processing method of the present invention, it is possible to more appropriately perform the gradation conversion processing on the image signal according to the dynamic range of the subject.

  Embodiments of the present invention will be described below with reference to the drawings.

[Embodiment 1]
1 to 11 show Embodiment 1 of the present invention. FIG. 1 is a block diagram showing a configuration of an imaging system to which an image signal processing apparatus is applied, and FIG. 2 is a block diagram showing a configuration of a correction unit. 3 is a diagram showing gradation conversion characteristics before correction, FIG. 4 is a diagram showing reference gradation conversion characteristics, FIG. 5 is a diagram showing gradation conversion characteristics after correction, and FIG. FIG. 7 is a diagram showing a pixel region obtained by comparison with FIG. 7, FIG. 7 is a diagram showing a state in which only the feature region which is an area equal to or larger than the structural element is left on the pixel region of FIG. 6, and FIG. FIG. 9 is a diagram showing the histogram after clip processing, and FIG. 10 is a diagram showing the tone conversion characteristics calculated by accumulating the histograms of FIGS. 8 and 9 respectively. FIG. 11 shows the flow of processing by the image signal processing program. Which is a flow chart showing.

  As shown in FIG. 1, the imaging system is configured as a digital camera, for example, and includes a lens system 100, an aperture 101, an AF motor 102, a color filter 103, a CCD 104, an AE sensor 105, an A / D conversion unit 106, buffer 107, shooting control unit 108, signal processing unit 109, feature amount extraction unit 110, gradation conversion characteristic calculation unit 111, correction unit 112, gradation conversion unit 113, A compression unit 114, an output unit 115, a control unit 116, and an external I / F unit 117. Here, a unit including the AE sensor 105, the gradation conversion characteristic calculation unit 111, the correction unit 112, and the gradation conversion unit 113 is appropriately referred to as an image signal processing apparatus.

  The lens system 100 forms a subject light image on the imaging surface of the CCD 104.

  The diaphragm 101 regulates the brightness of a subject light image formed on the imaging surface of the CCD 104 (by adjusting the amount of incident light on the imaging surface) by defining the passing range of the light flux from the lens system 100. It is.

  The AF motor 102 is a drive source for driving the lens system 100 to perform autofocus.

  The color filter 103 is configured as, for example, an RGB primary color filter with a Bayer arrangement (that is, a single-plate color CCD is assumed in the present embodiment).

  The CCD 104 photoelectrically converts the formed subject light image and outputs it as an analog image signal. The CCD 104 is configured so that the amount of light incident on the imaging surface can be adjusted by adjusting the exposure time. The lens system 100, the diaphragm 101, the AF motor 102, the color filter 103, the CCD 104, and the A / D conversion unit 106 constitute an imaging unit.

  The CCD 104 is connected to the output unit 115 via an A / D conversion unit 106, a buffer 107, a signal processing unit 109, a gradation conversion unit 113, and a compression unit 114 in this order. The buffer 107 is connected to the imaging control unit 108. The imaging control unit 108 is connected to the aperture 101, the AF motor 102, and the CCD 104. The AE sensor 105 is connected to the imaging control unit 108. The signal processing unit 109 is also connected to the feature amount extraction unit 110 and the gradation conversion characteristic calculation unit 111. The feature quantity extraction unit 110 and the gradation conversion characteristic calculation unit 111 are connected to the correction unit 112. The correction unit 112 is connected to the gradation conversion unit 113.

  The control unit 116 includes a photographing control unit 108, a signal processing unit 109, a feature amount extraction unit 110, a gradation conversion characteristic calculation unit 111, a correction unit 112, a gradation conversion unit 113, a compression unit 114, an external I / F unit 117, and the like. They are connected in both directions to control them. Here, the control unit 116 includes a microcomputer and the like.

  The external I / F unit 117 also includes a power switch for turning on / off the power of the image signal processing device, a shutter button including a two-stage press button for inputting an image capturing operation, and various modes during photographing. The interface includes a mode switch for switching (including exposure conditions such as shutter speed and aperture value, or ISO sensitivity).

  Furthermore, the AE sensor 105 is a luminance range acquisition unit that measures the luminance of the subject, acquires information about the width of the luminance range of the subject, and outputs the information. The range of the luminance range of the subject that can be measured by the AE sensor is sufficiently larger than the width of the dynamic range of the image captured by the CCD 104.

  Next, the operation of the image signal processing apparatus configured as shown in FIG. 1 will be described along the signal flow.

  After shooting conditions such as ISO sensitivity and exposure are set via the external I / F unit 117, when the shutter button of the external I / F unit 117 is half-pressed, the image signal processing apparatus enters the pre-shooting mode. enter.

  Then, the image signal photographed through the lens system 100, the aperture 101, the color filter 103, and the CCD 104 is converted into a digital signal by the A / D conversion unit 106, transferred to the buffer 107, and temporarily stored.

  The image signal in the buffer 107 is transferred to the photographing control unit 108.

  The imaging control unit 108 detects contrast information in the AF area in the image signal, and controls the AF motor 102 so that the contrast information is maximized. When the control is performed so that the contrast information is maximized, the imaging control unit 108 acquires distance information (focusing condition information) to the focused subject based on the driving state of the AF motor 102. A focus signal indicating that focusing has been performed and the acquired distance information are transmitted to the control unit 116. Here, focusing is performed by contrast AF (so-called hill-climbing AF), but the present invention is not limited to this. For example, the distance to the main subject is measured by using an infrared light emitting diode, an infrared sensor, or the like. The AF motor 102 may be controlled to perform focusing according to the distance.

  Further, the imaging control unit 108 controls the diaphragm 101 to adjust the amount of incident light reaching the CCD 104 based on the range of the luminance range of the subject acquired from the AE sensor 105, or the electronic shutter speed of the CCD 104. To control. Then, the imaging control unit 108 transmits the exposure condition information used for the control to the control unit 116.

  When the operation in the pre-photographing mode is performed, when the shutter button of the external I / F unit 117 is fully pressed next, the image signal processing apparatus enters the main photographing mode and will be described below. The actual shooting operation is performed.

  At the time of the actual photographing, information at the time of photographing (information on focusing condition and exposure condition) obtained by the photographing control unit 108 is transferred to the control unit 116. Then, the actual photographing is performed by the control unit 116 controlling the image signal processing apparatus using the information at the time of photographing.

  The image signal acquired by the actual photographing is converted into a digital signal by the A / D conversion unit 106, transferred to the buffer 107, and stored.

  The image signal in the buffer 107 is transferred to the signal processing unit 109.

  Based on the control of the control unit 116, the signal processing unit 109 reads a single-panel image signal from the buffer 107 and performs known interpolation processing, white balance processing, and the like, so that each of the three primary colors of RGB is obtained for each pixel. An image signal in a three-plate state in which the signals are aligned is generated.

The signal processing unit 109 may output the generated three-plate state image signal as it is, or may further output the RGB signal after converting it to a YCbCr signal as shown in the following Equation 1. good.
[Equation 1]

  The image signal processed by the signal processing unit 109 is transferred to the feature amount extraction unit 110, the gradation conversion characteristic calculation unit 111, and the gradation conversion unit 113.

  The feature amount extraction unit 110 extracts feature amounts from the image signal transferred from the signal processing unit 109, and selects the extracted feature amounts to the correction unit 112. The feature amount extraction by the feature amount extraction unit 110 will be described with reference to FIGS.

  The feature amount extraction unit 110 first sets a range of pixel values for the image signal, specifically, a specific color range such as skin color or green (range setting unit), and the pixel value falls within the set range. For example, a spatially continuous region constituted by the pixels to be extracted is extracted as shown in FIG.

  Next, the feature quantity extraction unit 110 performs an opening process as shown in FIG. 7 to remove a region smaller than a predetermined structural element.

  Then, the feature amount extraction unit 110 calculates an area of each region (referred to as a feature region) remaining after the opening process (area calculation unit), and uses the information regarding the calculated area as a feature amount to the correction unit 112. Forward. Here, when there are a plurality of extracted regions, the information related to the area transferred by the feature amount extraction unit 110 as the feature amount may be the sum of the areas of the plurality of regions, or a plurality of information. Of these regions, the area of the largest region may be used.

  The region extraction method by the feature amount extraction unit 110 is not limited to the above-described example. For example, the feature amount extraction unit 110 is configured to include a face region extraction unit and an area calculation unit, and this face region extraction is performed. Alternatively, a method may be used in which a face area as a feature area is extracted using a known face detection technique by the section, and the area of the extracted face area is calculated by the area calculation section.

  On the other hand, the gradation conversion characteristic calculation unit 111 calculates the gradation conversion characteristic for the image signal transferred from the signal processing unit 109. The gradation conversion characteristic calculation unit 111 may use predetermined gradation conversion characteristics when calculating the gradation conversion characteristics, or the gradation conversion directly calculated from the transferred image signal. You may make it use a characteristic.

  The calculation of the gradation conversion characteristic based on the histogram by the gradation conversion characteristic calculation unit 111 will be described with reference to FIGS.

  The gradation conversion characteristic calculation unit 111 first calculates a histogram based on the image signal transferred from the signal processing unit 109 (histogram calculation unit). This histogram is assumed to be as shown in FIG. 8, for example. Subsequently, the gradation conversion characteristic calculation unit 111 performs clip processing on the histogram with the clip values shown in FIG. 8 (clip unit). Then, the histogram after clip processing is as shown in FIG. Then, the gradation conversion characteristic calculation unit 111 calculates the gradation conversion characteristic as shown by the solid line in FIG. 10 by accumulating the histogram before the clipping process shown in FIG. 8, or shown in FIG. The gradation after the clip process is accumulated to calculate gradation conversion characteristics as indicated by the dotted line in FIG. 10 (accumulated value calculation unit). Here, the gradation conversion characteristic calculation unit 111 corrects the calculated gradation conversion characteristic by calculating either the gradation conversion characteristic indicated by the dotted line in FIG. 10 or the gradation conversion characteristic indicated by the solid line in FIG. The data is transferred to the unit 112.

  Next, an example of the configuration of the correction unit 112 will be described with reference to FIG.

  The correction unit 112 includes a ROM 200, a reference gradation conversion characteristic setting unit 201, and a weighted addition averaging unit 202.

  The ROM 200 is connected to the reference gradation conversion characteristic setting unit 201. The reference gradation conversion characteristic setting unit 201, the feature amount extraction unit 110, and the gradation conversion characteristic calculation unit 111 are connected to the weighted addition averaging unit 202. The weighted addition averaging unit 202 is connected to the gradation conversion unit 113. The control unit 116 is bi-directionally connected to the weighted addition averaging unit 202 and controls this.

  The ROM 200 stores a reference gradation conversion characteristic (for example, a reference gradation conversion characteristic as shown in FIG. 4. Here, the reference gradation conversion characteristic shown in FIG. In the case where the bit width of the luminance value and the bit width of the output luminance value are the same, the gradation conversion characteristics such that output luminance value = input luminance) is stored in a nonvolatile manner. Here, the reference tone conversion characteristic is a target tone conversion characteristic when the brightness range of the subject is large. Alternatively, it can also be said that the reference gradation conversion characteristic is a gradation conversion characteristic when the brightness range of the subject reaches the maximum.

  The reference gradation conversion characteristic setting unit 201 reads the reference gradation conversion characteristic from the ROM 200 and transfers the read gradation conversion characteristic to the weighted addition averaging unit 202.

  On the other hand, from the gradation conversion characteristic calculation unit 111, for example, the gradation conversion characteristic as illustrated in FIG. 3 is transferred to the weighted addition averaging unit 202.

ここに、Ｔ’（ｉ）は補正された階調変換特性、Ｔ（ｉ）は階調変換特性算出部１１１から転送されてきた階調変換特性（図３参照）、Ｂ（ｉ）は基準階調変換特性（図４参照）、をそれぞれ表している。 The weighted addition averaging unit 202 uses the gradation conversion characteristics as shown in FIG. 3 and the reference gradation conversion characteristics as shown in FIG. 4 to perform corrected gradation conversion as shown by the dotted line in FIG. The characteristic is calculated as shown in the following formula 2, for example.
[Equation 2]

Here, T ′ (i) is the corrected gradation conversion characteristic, T (i) is the gradation conversion characteristic transferred from the gradation conversion characteristic calculation unit 111 (see FIG. 3), and B (i) is the reference. Each represents gradation conversion characteristics (see FIG. 4).

ここに、ａは所定の係数を表している。 Further, α appearing on the right side of Equation 2 is a weighting coefficient that satisfies 0 ≦ α ≦ 1, and is transferred from the AE sensor 105 via the area S of the region transferred from the feature amount extraction unit 110 and the control unit 116. For example, the following equation 3 is calculated based on the brightness range ΔEV of the subject.
[Equation 3]

Here, a represents a predetermined coefficient.

  As can be seen from Equations 2 and 3, when the brightness range ΔEV of the subject is large, the gradation conversion characteristic T ′ (i) that is corrected because the weighting coefficient α is small is the reference gradation conversion. When the area S of the region extracted by the feature amount extraction unit 110 is close to the characteristic B (i) and the weighting coefficient α approaches 1, the gradation conversion characteristic T ′ (i) is corrected. Is close to the gradation conversion characteristic T (i) transferred from the gradation conversion characteristic calculation unit 111.

  In other words, when the brightness range ΔEV of the subject is large, it can be estimated that the image signal transferred from the signal processing unit 109 has many blackouts or whiteouts. If a gradation conversion process that greatly changes is performed, side effects such as amplifying the noise in the dark portion are considered to increase. Therefore, in this case, it is preferable to apply a gradation conversion characteristic that does not greatly change the output with respect to the input, and the gradation conversion characteristic T ′ (i) close to the reference gradation conversion characteristic B (i) is as described above. Is applied to the gradation conversion unit 113 in the subsequent stage.

  On the other hand, when the brightness range ΔEV of the subject is small, it can be estimated that the image signal transferred from the signal processing unit 109 has relatively little (or no) blackening or whiteout. There is no problem even if gradation conversion is performed with a large change in output relative to the input. Therefore, the gradation conversion characteristic T ′ (i) close to the gradation conversion characteristic T (i) transferred from the gradation conversion characteristic calculation unit 111 is applied in the subsequent gradation conversion unit 113, thereby improving the image quality. I try to get a big effect.

  In addition, when the area S of the feature region extracted from the image signal is large, it is considered that this feature region greatly contributes to the gradation conversion characteristics obtained by the accumulation of the histogram. Therefore, in order to keep the gradation of this characteristic region good, it is desirable to retain the gradation conversion characteristics obtained by the accumulation of the histogram to some extent. Therefore, the corrected gradation conversion characteristic T ′ (i) is made to approach the gradation conversion characteristic T (i) transferred from the gradation conversion characteristic calculation unit 111.

  On the other hand, when the area S of the feature region extracted from the image signal is small, it is considered that this feature region does not contribute much to the gradation conversion characteristics obtained by the accumulation of the histogram. Therefore, it is not necessary to particularly retain the gradation conversion characteristics obtained by the accumulation of the histogram, and a corrected gradation conversion characteristic T ′ (i) that is appropriate according to the width ΔEV of the subject luminance range can be used. Will be calculated.

  Note that the coefficient a appearing in Equation 3 may have a different value depending on what region the feature amount extraction unit 110 extracts. For example, when the region extracted by the feature quantity extraction unit 110 is a face region, it is conceivable to increase the value of the coefficient a in order to increase the contribution of the area S of the region.

  The gradation conversion characteristic T ′ (i) corrected by the correction unit 112 in this way is transferred to the gradation conversion unit 113.

  The gradation conversion unit 113 performs gradation conversion processing on the image signal transferred from the signal processing unit 109 using the gradation conversion characteristic T ′ (i) transferred from the correction unit 112. The gradation conversion unit 113 also performs gamma conversion in consideration of a display system such as a display. The image signal subjected to gradation conversion processing by the gradation conversion unit 113 is transferred to the compression unit 114.

  The compression unit 114 performs a known JPEG compression process on the image signal transferred from the gradation conversion unit 113, and transfers the compressed image signal to the output unit 115.

  The output unit 115 records and stores the compressed image signal transferred from the compression unit 114 in a recording medium such as a memory card. Alternatively, the output unit 115 may convert an uncompressed image signal into a video signal for display and display it for display on an external display or the like.

  In the above description, hardware processing is assumed. However, the present invention is not limited to this. For example, an image signal (a plurality of image signals having different exposure conditions) obtained from the CCD 104 obtained via the A / D conversion unit 106 is used as raw data as raw data, and the aperture value associated with each image signal in the raw data. Information such as exposure time, ISO sensitivity information, information on the range of the luminance range of the subject from the AE sensor 105, and video signal size are added as header information and output. It is also possible to input this data into a computer and cause the computer to execute an image signal processing program which is separate software for processing. In addition, instead of the imaging system, the image signal processing apparatus, or the image signal processing program, an image signal processing method that performs similar processing may be used.

  With reference to FIG. 11, the flow of processing by the image signal processing program will be described. Since the processing flow shown in FIG. 11 substantially corresponds to the processing flow in FIG. 1, the corresponding portions in the configuration shown in FIG. 1 are described as appropriate, and detailed description of the flow is omitted. To do.

  When this process is started, first, header information is read (step S1), and an image signal is input (step S2).

  Next, predetermined signal processing corresponding to the signal processing unit 109 is performed (step S3).

  Subsequently, while performing a gradation conversion characteristic calculation process corresponding to the gradation conversion characteristic calculation unit 111 (step S4), a process of extracting a feature region from the image signal corresponding to the feature amount extraction unit 110 is also performed (step S4). S5).

  Then, a gradation conversion characteristic correction process corresponding to the correction unit 112 is performed (step S6), and a gradation conversion process corresponding to the gradation conversion unit 113 is performed (step S7).

  Thereafter, it is determined whether or not the processing for all the target pixels has been completed (step S8). If it is determined that the processing has not ended, the processing returns to step S7. The processing by this image signal processing program is terminated.

  According to the first embodiment, since the dynamic range of the subject is measured and the tone conversion characteristics are corrected according to the measurement result, the adaptive tone conversion processing according to the dynamic range of the subject is performed. Thus, a higher quality video signal can be obtained.

  Since the measurement of the dynamic range of the subject is performed using the AE sensor 105 having a wider dynamic range than the image sensor, more accurate estimation can be performed.

  In addition, since the correction amount of the gradation conversion characteristic is varied according to the feature amount included in the image signal (for example, the area of the specific color region in the above), a characteristic such as a face region is included in the image. When an area is included, it is possible to perform gradation conversion while retaining the gradation of this characteristic area as much as possible.

[Embodiment 2]
12 and 13 show Embodiment 2 of the present invention. FIG. 12 is a block diagram showing the configuration of an imaging system to which the image signal processing apparatus is applied. FIG. 13 is a flow of processing by the image signal processing program. It is a flowchart which shows.

  In the second embodiment, parts that are the same as those in the first embodiment are given the same reference numerals and description thereof is omitted, and only differences are mainly described.

  The image signal processing apparatus according to the present embodiment is the same as the image signal processing apparatus shown in FIG. It is a configuration that is replaced with. The feature quantity extraction unit 110 and the gradation conversion characteristic calculation unit 111 are connected to the stop unit 1000. The stop unit 1000 is connected to the gradation conversion unit 113. The control unit 116 is also connected to the stop unit 1000 in both directions, and controls this.

  Next, portions of the operation of the image signal processing device of the present embodiment that are different from the operation of the image signal processing device of the first embodiment will be described along the signal flow in FIG.

  The size ΔEV of the luminance range of the subject acquired by the AE sensor 105, the area S of the feature region extracted by the feature amount extraction unit 110, and the gradation conversion characteristic T (i calculated by the gradation conversion characteristic calculation unit 111 ) Is transferred to the stopping unit 1000.

ここに、ｂは所定の係数を表している。 The stopping unit 1000 calculates β as shown in the following Equation 4 using ΔEV, S transferred.
[Equation 4]

Here, b represents a predetermined coefficient.

  Then, the stopping unit 1000 determines whether or not to perform gradation conversion processing based on the gradation conversion characteristic T (i) based on the calculated β. In other words, the stopping unit 1000 compares the calculated β with the threshold th, and when β ≦ th (this is because the area of the acquired luminance range of the subject is predetermined when the area S is fixed) If it is greater than or equal to the threshold value), it is determined that the gradation conversion processing is to be stopped, and if β> th (similarly, when the area S is fixed, the width of the acquired luminance range of the subject Is less than a predetermined threshold value), it is determined that the gradation conversion process based on the gradation conversion characteristic T (i) is to be executed, and information indicating the determination result is transferred to the gradation conversion unit 113. At the same time, when the determination result is a result of executing the gradation conversion process, the gradation conversion characteristic T (i) is also transferred to the gradation conversion unit 113.

  Here, the stopping unit 1000 uses β calculated as shown in Equation 4 in determining whether to perform the gradation conversion processing, but instead of using this β, the above-described embodiment is used. Α as shown in Formula 3 of 1 may be used. Also in this case, the stopping unit 1000 makes a determination by comparing α with an appropriate threshold value.

  When the determination result transferred from the stop unit is to perform tone conversion, the tone conversion unit 113 uses the tone conversion characteristic T (i) and is transferred from the signal processing unit 109. A gradation conversion process is performed on the image signal. On the other hand, when the determination result transferred from the stop unit does not perform tone conversion, the tone conversion unit 113 performs tone conversion processing on the image signal transferred from the signal processing unit 109. Absent. However, regardless of the determination result, the gradation conversion unit 113 performs gamma conversion in consideration of a display system such as a display.

  In the above description, hardware processing is assumed. However, the present invention is not limited to this, and the image signal processing program can be executed by a computer and processed as in the first embodiment. However, an image signal processing method that performs the same processing may be used.

  With reference to FIG. 13, the flow of processing by the image signal processing program will be described. Since the processing flow shown in FIG. 13 substantially corresponds to the processing flow in FIG. 12, the corresponding portions in the configuration shown in FIG. 12 are described as appropriate, and detailed description of the flow is omitted. To do. Also, the description of the same parts as those shown in FIG. 11 of the first embodiment will be omitted.

  When the process of step S5 has been performed (note that the process of step S4 does not have to be already performed at this point, it is sufficient if it is completed before performing the subsequent step S7). A determination process for determining whether or not to stop the gradation conversion process is performed (step S20).

  In this determination process, when it is determined that the gradation conversion process is to be performed, the gradation in step S7 corresponding to the gradation conversion unit 113 using the gradation conversion characteristic T (i) calculated in step S4. The conversion processing is performed until it is determined in step S8 that the processing for all the target pixels has been completed. On the other hand, when it is determined that the gradation conversion processing is not performed, the processing in steps S7 and S8 is skipped. To exit. Other processes are the same as those in FIG.

  According to the second embodiment, since the dynamic range of the subject is measured and it is determined whether or not the gradation conversion is performed according to the measurement result, the image signal is processed even if the gradation conversion process is performed. When it is estimated that almost no improvement in image quality can be obtained, the tone conversion process can be omitted, so that the processing speed can be increased.

  Furthermore, since it is determined whether or not to perform gradation conversion based on the feature amount included in the image signal, it is possible to make an appropriate determination according to the feature amount.

  In addition, according to the second embodiment, substantially the same effects as those of the first embodiment described above can be achieved.

  Note that the gradation conversion characteristic calculation unit 111 calculates the gradation conversion characteristic based on the histogram of the image, so even if the images are completely different, the same gradation conversion characteristic is calculated if the histograms are the same. Will be. Therefore, if a separate nonvolatile memory or the like is provided and the histogram and the gradation conversion characteristic are stored in association with each other in this nonvolatile memory, the gradation conversion characteristic is newly added when the histograms of the images are the same. It is sufficient to search and output already stored gradation conversion characteristics without calculation.

  Also, if the brightness range, the histogram, and the gradation conversion characteristics are associated with each other and stored in the nonvolatile memory, the same brightness range width and histogram image signal as the already stored gradation conversion characteristics are stored. When performing this processing, it may be possible to omit correction by the correction unit 112 (for example, when correction by a feature amount is omitted).

  Furthermore, if the non-volatile memory is stored by associating the brightness range, the feature amount, the histogram, and the tone conversion characteristics, the same brightness range size and characteristics as the already stored tone conversion characteristics. When the amount and the image signal of the histogram are processed, the correction by the correction unit 112 can be omitted.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, you may delete a some component from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

1 is a block diagram showing a configuration of an imaging system to which an image signal processing device according to Embodiment 1 of the present invention is applied. FIG. 3 is a block diagram showing a configuration of a correction unit in the first embodiment. FIG. 3 is a diagram illustrating gradation conversion characteristics before correction in the first embodiment. FIG. 3 is a diagram showing reference gradation conversion characteristics in the first embodiment. FIG. 3 is a diagram showing gradation conversion characteristics after correction in the first embodiment. The figure which shows the area | region of the pixel obtained in the said Embodiment 1 by comparing an image with a threshold value. FIG. 7 is a diagram illustrating a state where an opening process is performed on the pixel region in FIG. 6 and only a feature region that is a region larger than a structural element is left in the first embodiment. In the said Embodiment 1, the diagram which shows the histogram and clip value before a clip process. The diagram which shows the histogram after the clip process in the said Embodiment 1. FIG. FIG. 10 is a diagram showing a state of gradation conversion characteristics calculated by accumulating the histograms of FIGS. 8 and 9 in the first embodiment. 3 is a flowchart showing a flow of processing by an image signal processing program in the first embodiment. The block diagram which shows the structure of the imaging system with which the image signal processing apparatus in Embodiment 2 of this invention is applied. 9 is a flowchart showing a flow of processing by an image signal processing program in the second embodiment.

Explanation of symbols

100: Lens system (imaging unit)
101 ... Aperture (imaging part)
102: AF motor (imaging unit)
103. Color filter (imaging unit)
104 ... CCD (imaging part)
105... AE sensor (luminance range acquisition unit)
106... A / D converter (imaging unit)
DESCRIPTION OF SYMBOLS 107 ... Buffer 108 ... Shooting control part 109 ... Signal processing part 110 ... Feature-value extraction part 111 ... Gradation conversion characteristic calculation part 112 ... Correction | amendment part 113 ... Gradation conversion part 114 ... Compression part 115 ... Output part 116 ... Control part 117 ... External I / F part 200 ... ROM
201: reference gradation conversion characteristic setting unit 202 ... weighted averaging unit 1000 ... stop unit

Claims (12)

An image signal processing apparatus capable of gradation conversion of an image signal,
A luminance range acquisition unit that measures the luminance of the subject and acquires the width of the luminance range of the subject;
A gradation conversion characteristic calculator that calculates gradation conversion characteristics based on pixel values of pixels included in an image signal of an image in which the subject is captured;
Based on the width of the luminance range acquired by the luminance range acquisition unit, when the width of the luminance range is relatively small, it is calculated by the gradation conversion characteristic calculation unit than when it is relatively large A correction unit for correcting the gradation conversion characteristics so as to approach the gradation conversion characteristics;
A gradation conversion unit that performs gradation conversion of the image signal using the gradation conversion characteristic corrected by the correction unit;
An image signal processing apparatus comprising: The correction unit is
A reference gradation conversion characteristic setting unit for setting a reference gradation conversion characteristic;
When the acquired brightness range of the subject is relatively small, it approaches the gradation conversion characteristic calculated by the gradation conversion characteristic calculation unit, and when it is relatively large, the reference gradation conversion is performed. A weighted addition averaging unit that calculates a gradation conversion characteristic corrected by weighted addition averaging the gradation conversion characteristic and the reference gradation conversion characteristic so as to approach the characteristic;
The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus is configured to include:   The image signal processing apparatus according to claim 2, wherein the reference gradation conversion characteristic is a gradation conversion characteristic in which an input pixel value and an output pixel value have a linear relationship. The correction unit includes a stop unit that stops the gradation conversion process by the gradation conversion unit when the width of the acquired luminance range of the subject is equal to or greater than a predetermined threshold,
The gradation conversion process by the gradation conversion unit based on the gradation conversion characteristic calculated by the gradation conversion characteristic calculation unit is performed only when the acquired brightness range of the subject is less than a predetermined threshold. The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus is allowed. The gradation conversion characteristic calculation unit
A histogram calculation unit for calculating a histogram from the image signal;
A clip unit that performs clip processing on the histogram;
A cumulative value calculation unit for calculating the gradation conversion characteristics by calculating a cumulative value of the histogram subjected to the clipping process;
5. The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus is configured to include: A feature amount extraction unit for extracting the feature amount of the image signal;
The correction unit corrects the gradation conversion characteristic based on the feature amount in addition to the width of the luminance range.
The image signal processing apparatus according to claim 1, wherein the image signal processing apparatus is an image signal processing apparatus. The feature quantity extraction unit
A region extraction unit for extracting a feature region in the image signal;
An area calculation unit for calculating the area of the feature region as the feature amount;
The image signal processing apparatus according to claim 6, comprising: The feature amount extraction unit further includes a range setting unit that sets a range of pixel values,
The region extraction unit extracts a spatially continuous region occupied by pixels having pixel values within a range set by the range setting unit as the feature region. Item 8. The image signal processing device according to Item 7. The correction unit is
A reference gradation conversion characteristic setting unit for setting a reference gradation conversion characteristic;
When the acquired brightness range of the subject is relatively small, it approaches the gradation conversion characteristic calculated by the gradation conversion characteristic calculation unit, and when it is relatively large, the reference gradation conversion is performed. If the area of the feature region is relatively large so as to approach the characteristic, the gradation conversion characteristic calculated by the gradation conversion characteristic calculator is close, and if the area is relatively small, the reference floor A weighted addition averaging unit that calculates a gradation conversion characteristic corrected by weighted addition averaging the gradation conversion characteristic and the reference gradation conversion characteristic so as to approach the tone conversion characteristic;
The image signal processing apparatus according to claim 7 or 8, wherein the image signal processing apparatus is configured to include: An imaging system capable of gradation conversion of an image signal,
An imaging unit for imaging a subject and outputting an image signal;
A luminance range acquisition unit that measures the luminance of the subject imaged by the imaging unit and acquires the width of the luminance range of the subject;
A gradation conversion characteristic calculator that calculates gradation conversion characteristics based on pixel values of pixels included in an image signal of an image in which the subject is captured;
Based on the width of the luminance range acquired by the luminance range acquisition unit, when the width of the luminance range is relatively small, it is calculated by the gradation conversion characteristic calculation unit than when it is relatively large A correction unit for correcting the gradation conversion characteristics so as to approach the gradation conversion characteristics;
A gradation conversion unit that performs gradation conversion of the image signal using the gradation conversion characteristic corrected by the correction unit;
An imaging system comprising: An image signal processing program for causing a computer to perform gradation conversion of an image signal, wherein the computer
A gradation conversion characteristic calculating step for calculating a gradation conversion characteristic based on a pixel value of a pixel included in the image signal;
Based on the brightness range of the subject acquired at the time of imaging, the level calculated by the gradation conversion characteristic calculation step is larger when the brightness range is relatively smaller than when the brightness range is relatively large. A correction step for correcting the tone conversion characteristics so as to approach the tone conversion characteristics;
A gradation conversion step for gradation-converting the image signal using the gradation conversion characteristics corrected by the correction step;
Image signal processing program for executing An image signal processing method for gradation conversion of an image signal,
A gradation conversion characteristic calculating step for calculating a gradation conversion characteristic based on a pixel value of a pixel included in the image signal;
Based on the brightness range of the subject acquired at the time of imaging, the level calculated by the gradation conversion characteristic calculation step is larger when the brightness range is relatively smaller than when the brightness range is relatively large. A correction step for correcting the tone conversion characteristics so as to approach the tone conversion characteristics;
A gradation conversion step for gradation-converting the image signal using the gradation conversion characteristics corrected by the correction step;
An image signal processing method comprising:

Images