JP2014096752A - Image processor, image processing method and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor which can more properly compress a dynamic range by using histogram equalization even if there is deviation of frequency in a histogram and to provide an image processing method and an imaging apparatus.SOLUTION: An image processor: generates a histogram of pixel values of individual pixels in an input image; generates offset values of individual classes, which increase monotonously in accordance with an increase of the classes, for the individual classes from the minimum class having the frequency to the maximum class having the frequency in the generated histogram; adds the generated offset values of the individual classes respectively to the frequencies of the classes from the minimum class to the maximum class; generates a compression characteristic of the input image on the basis of the histogram after the offset values are added; and generates a compression image whose dynamic range is smaller than that of the input image by using the compression characteristic.

Description

  The present invention relates to an image processing apparatus and an image processing method for compressing a dynamic range of an image using a histogram equalization method, and an imaging apparatus using the image processing apparatus.

  In recent years, in an imaging apparatus applied to a digital camera such as a digital still camera or a digital video camera, the luminance range of a subject (difference between the lowest luminance and the highest luminance, the lowest density in an image) One of the themes is to expand the dynamic range (DR), that is, the difference from the maximum density. As a technique for expanding the dynamic range, various techniques such as a so-called linear log sensor have been researched and developed.

  While the expansion of the dynamic range in this imaging device has progressed, the dynamic range (the number of bits representing the pixel level) in a device that transmits, stores, or displays an image is currently relative to the dynamic range in the imaging device. It may be narrow. In such a case, even if the dynamic range in the imaging apparatus is expanded, it is difficult to handle all the obtained information. For this reason, a dynamic range compression technique for converting a wide dynamic range image into a narrower dynamic range image has been researched and developed.

  As this dynamic range compression technique, for example, as disclosed in Patent Document 1 and Patent Document 2, a first method for performing frequency processing and a second method for performing histogram equalization are known. In this first method, an image to be compressed is first divided into a high frequency component and a low frequency component, and then the low frequency component is compressed, and the compressed low frequency component is converted into the high frequency component. Ingredients are added. This compresses the dynamic range of the image. In the second method, first, a histogram (frequency distribution) in the image to be compressed is obtained, then a cumulative frequency distribution is obtained from the histogram of the image, and then the vertical axis of the cumulative frequency distribution is compressed. The pixel values of the image are converted by using the normalized cumulative frequency distribution as a compression conversion characteristic. This compresses the dynamic range of the image.

  On the other hand, histogram equalization is also known as a technique for optimizing the contrast of an image, and is disclosed in, for example, Patent Documents 3 to 5. The image contrast processing apparatus disclosed in Patent Document 3 calculates a difference between integral values of a histogram for a target image, a means for compressing the histogram in the pixel number direction, and a histogram before and after compression. Means, a means for creating a new histogram in which the average value of the calculated values is uniformly applied as a bias to each gradation of the compressed histogram, and the above target based on the new histogram. Means for performing gradation conversion for histogram equalization on an image of the object.

  The X-ray image processing apparatus disclosed in Patent Document 4 is an X-ray image processing apparatus provided with window processing means for performing gradation conversion on the processing target image data received from the image data supply means using a window. The X-ray diaphragm and the halation gradation area are extracted from the histogram of the processing target image data, and a histogram only for the gradation area of the subject other than the X-ray diaphragm and the halation gradation area is created. Gradation conversion control means for performing gradation conversion in the window processing means by a window in which the created histogram is transformed into a predetermined shape is provided.

  Further, an image processing device disclosed in Patent Document 5 includes an average deviation calculating unit that calculates an average and a deviation of pixel values constituting input image data, and a deviation of the pixel value centered on the average of the pixel values. Histogram generation means for generating a histogram of the pixel values by truncating pixels having values outside the predetermined range, and conversion means for converting the pixel values using the histogram generated by the histogram generation means. ing.

JP-A-7-306938 Japanese Patent Laid-Open No. 2003-8898 JP-A-1-222382 Japanese Patent Laid-Open No. 5-84235 JP 2003-308529 A

  By the way, the dynamic range compression technique using the above-mentioned histogram equalization method is good when the histogram of the image is a substantially flat profile, that is, when the cumulative frequency distribution of the image is a profile that changes substantially linearly. An image with a compressed range can be generated. However, when the frequencies in the histogram are unevenly distributed, such as an image obtained by capturing a night view or almost darkness, an unnatural image is caused by the pixel value converted by the compression conversion characteristic being too large. Become.

  The present invention has been made in view of the above circumstances, and its purpose is to compress the dynamic range more appropriately using the histogram equalization method even when the frequencies in the histogram are unevenly distributed. An image processing apparatus, an image processing method, and an imaging apparatus using the same are provided.

  As a result of various studies, the present inventor has found that the above object is achieved by the present invention described below. That is, an image processing apparatus according to an aspect of the present invention includes a histogram generation unit that generates a histogram of pixel values of each pixel in an input image, and a frequency from the lowest class having a frequency in the histogram generated by the histogram generation unit. For each class up to the maximum class, an offset value for each class that increases monotonically as the class increases is generated, and the generated offset value for each class is generated from the minimum class to the maximum class. An offset processing unit that adds to the frequency of each class, a compression characteristic generation unit that generates a compression characteristic for compressing the input image based on the histogram after the offset value addition generated by the offset processing unit, By using the compression characteristic generated by the compression characteristic generation unit, the dynamic range is greater than that of the input image. Characterized in that it comprises a compression image generation unit that generates a small compressed images. Preferably, the compression characteristic generation unit generates a cumulative histogram from the histogram after addition of the offset value generated by the offset processing unit, and normalizes the generated cumulative histogram to a range of pixel values that can be taken by the compressed image. By doing so, a compression characteristic is generated.

  If the compression characteristics are generated using the histogram with the frequency unevenly distributed as it is, the gradation is preferentially assigned to the portion where the frequency is biased, resulting in an unnatural image. . In the image processing apparatus having the above configuration, the offset value of each class that monotonously increases as the class increases is added to the frequency of each class in the histogram. For this reason, in such an image processing apparatus, gradation assignment is performed throughout, and the glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be more appropriately compressed.

  In another aspect, in the image processing apparatus described above, the histogram generation unit generates the predetermined frequency by giving a predetermined frequency to a class larger than the maximum class having the frequency in the histogram generated by the histogram generation unit. The image processing apparatus further includes an expansion processing unit that expands a histogram in a class direction, and the offset processing unit is configured to perform processing from the lowest class having a frequency to the maximum class given the predetermined frequency in the histogram expanded by the expansion processing unit. For each class, an offset value for each class that increases monotonically as the class increases is generated, and the generated offset value for each class is set to the frequency of each class from the minimum class to the maximum class, respectively. It is characterized by adding.

  In such an image processing apparatus, the maximum class having the frequency in the histogram generated by the histogram generation unit becomes a larger class by the enlargement processing unit, and the frequency is given to the class originally having no frequency by the offset processing unit. It will be. Therefore, in such an image processing apparatus, even in the case of a histogram in which the frequencies are concentrated in a predetermined range of classes and the frequencies are unevenly distributed, gradation allocation is performed over a wider range, and histogram equalization is performed. The glare peculiar to the law can be suppressed, and the dynamic range can be compressed more appropriately.

  In another aspect, the image processing apparatus further includes a clip processing unit that limits a frequency to a predetermined limit value with respect to the histogram generated by the histogram generation unit, wherein the offset processing unit includes the clip processing unit. For each class from the lowest class with frequency to the highest class with frequency in the histogram processed by the processing unit, an offset value of each class that increases monotonically as the class increases is generated, and the generated The offset value of each class is added to the frequency of each class from the minimum class to the maximum class. In another aspect, the image processing apparatus further includes a clip processing unit that limits a frequency to a predetermined limit value with respect to the histogram enlarged by the enlargement processing unit, and the offset processing unit includes: In the histogram processed by the clip processing unit, for each class from the lowest class with frequency to the highest class with frequency, an offset value of each class that increases monotonically as the class increases is generated, and the generation The offset value of each class is added to the frequency of each class from the minimum class to the maximum class.

  In such an image processing apparatus, even in the case of a histogram in which frequencies are unevenly distributed, the frequency is limited to a predetermined limit value by the clip processing unit, so that the degree of frequency deviation with respect to the whole is reduced. For this reason, in such an image processing apparatus, gradation allocation is made uniform as a whole, the glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be more appropriately compressed.

  According to another aspect, in the above-described image processing apparatuses, the predetermined limit value is different for each class.

  In such an image processing apparatus, the predetermined limit value can be adjusted for each class, so that the gradation allocation can be adjusted to be uniformized as a whole. Therefore, in such an image processing apparatus, it is possible to further suppress the glare feeling peculiar to the histogram equalization method, and to more appropriately compress the dynamic range.

  Further, in another aspect, in the above-described image processing devices, the offset processing unit includes a plurality of pattern generation processing units that respectively generate offset values of each class that monotonously increase as the class increases in a plurality of patterns. It is further provided with the feature.

  In such an image processing apparatus, it is possible to generate a compressed image that is offset-processed with a plurality of patterns, and thus it is possible to select an optimal compressed image from various compressed images.

  An image processing method according to another aspect of the present invention includes a histogram generation step of generating a histogram of pixel values of each pixel in an input image, and a minimum class having a frequency in the histogram generated in the histogram generation step. For each class up to the maximum class with frequency, an offset value of each class that monotonously increases as the class increases is generated, and the offset value of each generated class is changed from the minimum class to the maximum class. An offset processing step of adding to the frequency of each class until, and a compression characteristic generation step of generating a compression characteristic for compressing the input image based on the histogram after the offset value addition generated in the offset processing step, By using the compression characteristic generated in the compression characteristic generation step, Characterized in that it comprises a compressed image generation step of generating a small compressed images dynamic range.

  In such an image processing method, the offset value of each class that monotonously increases as the class increases is added to the frequency of each class in the histogram. For this reason, in such an image processing method, gradation assignment is performed throughout, and the glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be more appropriately compressed.

  An imaging apparatus according to another aspect of the present invention includes: an imaging unit that photoelectrically converts a subject light image to generate an image signal; and an image obtained by performing predetermined image processing on the image signal generated by the imaging unit. And an image processing unit that generates any of the above-described image processing devices.

  In such an imaging apparatus, since the image processing unit includes any one of the above-described image processing apparatuses, the dynamic range can be more appropriately compressed.

  The image processing apparatus, the image processing method, and the imaging apparatus using the image processing apparatus according to the present invention can compress the dynamic range more appropriately using the histogram equalization method even when the frequencies in the histogram are unevenly distributed. .

It is a block diagram which shows the structure of the imaging device in embodiment. It is a block diagram which shows the structure of the dynamic range compression process part in the imaging device shown in FIG. As an example, it is a figure which shows the look-up table which shows the correspondence of a class and an offset coefficient. As an example, it is a diagram showing a look-up table showing a correspondence relationship between a maximum effective area number and a base bias value. It is a flowchart which shows operation | movement of the dynamic range compression process part shown in FIG. FIG. 6 is a diagram for describing a processing result generated in each processing step of the operation illustrated in FIG. 5 when the histogram of the input image has two peaks. FIG. 6 is a diagram for describing a processing result generated in each processing step of the operation illustrated in FIG. 5 when an input image is a night view. It is a figure for demonstrating the process result produced | generated at each process process of the operation | movement shown in FIG. 5, when an input image is darkness. When the histogram of an input image is not biased, it is a figure for demonstrating the process result produced | generated by each process process of the dynamic range compression technique which performs the conventional histogram equalization as a comparative example. When the histogram of an input image is biased, it is a figure for demonstrating the process result produced | generated in each process process of the dynamic range compression technique which performs the conventional histogram equalization as a comparative example.

  Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted suitably.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to the embodiment. FIG. 2 is a block diagram illustrating a configuration of a dynamic range compression processing unit in the imaging apparatus illustrated in FIG. FIG. 3 is a diagram showing a look-up table showing the correspondence between classes and offset coefficients as an example. FIG. 4 is a diagram illustrating a look-up table showing a correspondence relationship between the maximum effective area number and the base bias value as an example.

  In FIG. 1, the imaging apparatus CA in the embodiment includes an imaging optical system 1, an imaging sensor 2, and an image processing unit 3.

  The imaging optical system 1 includes one or a plurality of optical elements such as lenses, and forms an optical image of an object (subject) on the light receiving surface of the imaging sensor 2. The imaging optical system 1 may perform, for example, focusing or zooming (magnification) by moving a predetermined optical element of the one or more optical elements along the optical axis AX. . The imaging sensor 2 includes a plurality of photoelectric conversion elements arranged so as to form a two-dimensional planar light-receiving surface, and R (red) according to the amount of light in the optical image of the object imaged by the imaging optical system 1. This is an element that photoelectrically converts signals of G (green) and B (blue) components and outputs the signals to the image processing unit 3. The imaging sensor 2 is, for example, a CCD type image sensor, a CMOS type image sensor, or the like. In this embodiment, the imaging sensor 2 is a so-called linear log sensor having a wide dynamic range that can correspond to a wide luminance range of a subject. This linear log sensor outputs an electric signal linearly photoelectrically converted with respect to an incident light amount equal to or less than a predetermined threshold value (inflection point), and logarithmically photoelectrically converts an incident light amount larger than the predetermined threshold value. An image sensor that outputs an electrical signal. The optical image of the object is guided to the light receiving surface of the imaging sensor 2 along the optical axis AX by the imaging optical system 1, and the optical image of the object is captured by the imaging sensor 2.

  The image processing unit 3 generates a predetermined image signal from the signal input from the image sensor 2 by performing predetermined image processing on the signal input from the image sensor 2, and in the present embodiment, The image processing unit 3 generates a compression characteristic for compressing the dynamic range based on a histogram of pixel values of each pixel in the input image, and by using the generated compression characteristic, the dynamic range of the dynamic range is larger than that of the input image. Generate a small compressed image. In the present embodiment, such an image processing unit 3 includes, for example, a sensor output processing unit 31, a dynamic range compression processing unit (hereinafter abbreviated as “DRC processing unit”) 32, and a luminance chromaticity processing unit 33. And an NTSC processing unit 34 and an arithmetic processing unit 35.

  The sensor output processing unit 31 generates a signal that can be easily processed by the image processing unit 3 from the signal input from the imaging sensor 2 by performing predetermined preprocessing on the signal input from the imaging sensor 2. It is. In the present embodiment, since the imaging sensor 2 is the linear log sensor 2, the sensor output processing unit 31 performs so-called full log processing as the predetermined preprocessing in order to unify photoelectric conversion characteristics. The full log process is a process of converting the signal value of the signal that has been linearly converted by the linear log sensor 2 into the signal value of the signal that has been logarithmically converted. The signal input from the imaging sensor 2 becomes a signal value of a signal photoelectrically converted by logarithmic conversion over the entire incident light amount regardless of the incident light amount by this full log processing. The sensor output processing unit 31 may further perform amplification processing for amplifying the signal input from the image sensor 2 while adjusting the gain (amplification factor) by AGC (auto gain control) as the predetermined preprocessing. Well, CDS (correlated double sampling) processing may be further performed to reduce sampling noise. The processing result processed by the sensor output processing unit 31 is output to the DRC processing unit 32 and the arithmetic processing unit 35.

  The DRC processing unit 32 performs processing for compressing the dynamic range of the processing result processed by the sensor output processing unit 31. The DRC processing unit 32 will be described later. The processing result processed by the DRC processing unit 32 is output to the luminance / chromaticity processing unit 33.

  The luminance chromaticity processing unit 33 performs various adjustment processes such as luminance and chromaticity on the processing result processed by the DRC processing unit 32. The luminance chromaticity processing unit 33 performs, for example, black level correction processing, white balance correction processing, color interpolation processing, color correction processing, and gamma correction processing (γ correction processing) as the adjustment processing. Each of these processes is processed by a known processing method. The processing result processed by the luminance / chromaticity processing unit 33 is output to the NTSC processing unit 34.

  The NTSC processing unit 34 performs processing for converting the processing result processed by the luminance chromaticity processing unit 33 into a signal conforming to the NTSC (National Television System Committee) standard. The NTSC processing unit 34 outputs an image signal conforming to the NTSC standard to an unillustrated display as an NTSC output. An image is displayed on the display (not shown).

  The arithmetic processing unit 35 controls the entire operation of the image processing unit 3 by controlling each unit of the image processing unit 3 according to the function. In the present embodiment, for example, the arithmetic processing unit 35 outputs various setting data necessary for compressing the dynamic range by the DRC processing unit 32 to the DRC processing unit 32, and the luminance / chromaticity processing unit 33 performs the adjustment processing. Various setting data (for example, gamma setting data for gamma correction processing) necessary for the execution is output to the luminance / chromaticity processing unit 33.

  Such an image processing unit 3 may be configured by individual components for each of the processing units 31 to 35, for example. For example, the image processing unit 3 includes a DSP (Digital Signal Processor), a memory element, a peripheral circuit, and the like. The processing units 31 to 35 may be functionally configured in a DSP or the like by executing a predetermined program.

  Next, the DRC processing unit 32 will be described. In FIG. 2, the DRC processing unit 32 includes, for example, a histogram analysis unit 321, an enlargement processing unit 322, a clip processing unit 323, an offset processing unit 324, a cumulative histogram generation unit 325, a normalization processing unit 326, And a compressed image generation unit 327.

  The histogram analysis unit 321 generates a histogram of pixel values of each pixel using the processing result processed by the sensor output processing unit 31. This histogram divides the range of pixel values that a pixel can take, for example, the range of luminance values into multiple classes at equal intervals, and for each class, the sum of the pixels with pixel values of that class is handled as the frequency (frequency) It is the attached statistical graph. In the histogram, the horizontal axis is class and the vertical axis is frequency. Note that the pixel value (for example, luminance value) is 0 or more. By referring to such a histogram, it is possible to visually recognize the distribution of pixel values of each pixel in the input image. Then, the histogram analysis unit 321 performs predetermined analysis in order to obtain data necessary for performing predetermined processing according to the function of each processing unit in the enlargement processing unit 322, the clip processing unit 323, or the offset processing unit 324. Processing is also performed on the generated histogram. As this predetermined analysis processing, for example, the histogram analysis unit 321 detects the minimum class having the frequency (effective area minimum class) and the maximum class having the frequency (effective area maximum class), and further, from this detection result. , To detect the class range (effective area range) with frequency. The histogram analysis unit 321 outputs the generated histogram to the enlargement processing unit 322, and outputs the analysis result obtained by the predetermined analysis processing to the arithmetic processing unit 35. In the following description, a class having a frequency may be referred to as an effective area.

  When obtaining the analysis result, the arithmetic processing unit 35 may output the effective area minimum class, the effective area maximum class, and the effective area range to the enlargement processing unit 322 as one of the setting data, but in the present embodiment, When obtaining the analysis result, the arithmetic processing unit 35 outputs at least the effective area maximum class to the enlargement processing unit 322 as one of the setting data. When the arithmetic processing unit 35 obtains the analysis result, the arithmetic processing unit 35 obtains a base bias value used in the offset processing based on the analysis result, and uses the obtained base bias value as one of setting data to the offset processing unit 324. Output. This base offset value will be described later.

  The enlargement processing unit 322 gives the histogram generated by the histogram analysis unit 321 by giving a predetermined frequency to a class larger than the maximum class having the frequency in the histogram generated by the histogram analysis unit 321 (effective area maximum class). It expands in the direction. More specifically, the enlargement processing unit 322 uses a predetermined threshold in which the maximum class having the frequency in the histogram generated by the histogram analysis unit 321 (effective area maximum class input from the arithmetic processing unit 35) is set in advance. When it is small, the histogram generated by the histogram analysis unit 321 is expanded in the class direction by giving a preset frequency (dummy frequency) to the class including the predetermined threshold. The dummy frequency is a small number such as 1 or 2, for example, as long as the frequency exists in the class including the predetermined threshold. By the processing of the enlargement processing unit 322, the effective area range of the histogram generated by the histogram analysis unit 321 is expanded and expanded in the class direction. The enlargement processing unit 322 outputs the enlarged histogram to the clip processing unit 323.

  When the frequency of the histogram is biased to a class within a predetermined range, the predetermined threshold value is obtained by making the frequency of the histogram cover all over by the processing of the offset processing unit 324, so that the gradation allocation is made uniform throughout. It is set so as to suppress the sensation of glare peculiar to the equalization method. For example, the predetermined threshold is set to 3/5 or 3/4 of the maximum value (maximum class) that the pixel value can take.

  In the above description, the dummy frequency is given only to the class including the predetermined threshold. However, the enlargement processing unit 322 has a predetermined maximum class having the frequency in the histogram generated by the histogram analysis unit 321. In the case where the threshold value is smaller than the threshold value, a dummy frequency may be given to each class from the class one higher than the maximum class having the frequency in the histogram generated by the histogram analysis unit 321 to the class including the predetermined threshold. .

The clip processing unit 323 limits the frequency to a predetermined limit value for the histogram processed by the enlargement processing unit 322. More specifically, the clip processing unit 323 sets a limit value for each class by a function that monotonously increases as the class increases, and for each class, the frequency of the class exceeds the limit value of the class. If so, the frequency of the class is replaced with the limit value of the class. The function is, for example, a function that increases monotonically in an exponential manner as the class increases. When the class is x, the limit value is y, the constant a> 1, and n is a natural number, y = a ^x and y = x ²ⁿ may be satisfied. The clip processing unit 323 outputs the clip processed histogram to the offset processing unit 324.

  The offset processing unit 324 has an offset value for each class that increases monotonically as the class increases for each class from the lowest class having the frequency to the highest class having the frequency in the histogram processed by the clip processing unit 323. Are generated, and the generated offset value of each class is added to the frequency of each class from the minimum class to the maximum class. More specifically, the offset processing unit 324 multiplies the base bias value input from the arithmetic processing unit 35 by a coefficient (offset coefficient) set in advance for each class so as to increase monotonously as the class increases. Thus, an offset value of each class is generated, and the generated offset value of each class is added to the frequency of each class from the minimum class to the maximum class. The offset processing unit 324 outputs the offset processed histogram to the cumulative histogram generation unit 325.

  The base bias value is set based on, for example, the maximum class (maximum effective area number) having the frequency in the histogram generated by the histogram analysis unit 321. More specifically, first, a correspondence between a class and a base bias value is created in advance for each class, and this correspondence is stored in the image processing unit 3 as a correspondence expression or a correspondence table. Then, the base bias value corresponding to the maximum class having the frequency in the histogram generated by the histogram analysis unit 321 is searched from the corresponding relational expression or the correspondence relation table, whereby the base bias value for the input image is determined. .

  The correspondence between the offset coefficient and the maximum effective area number of each class and the base bias value is preliminarily made uniform by assigning tones throughout the trial image, and is unique to the histogram equalization method. It is set so that a feeling of swaying can be suppressed. As an example, the correspondence relationship between the maximum effective area number and the base bias value is set to monotonously decrease as the class increases.

  An example of each offset coefficient of each class is shown in FIG. 3, and an example of a correspondence relationship between the maximum effective area number and the base bias value is shown in FIG. In each example shown in FIGS. 3 and 4, the number of classes is 16, but the number of classes is not limited to this and may be arbitrary. An example of the operation of the offset processing unit 324 will be described with reference to the examples of FIGS. 3 and 4. If the maximum effective area number is 7, the offset processing unit 324 has the maximum effective area number and the base shown in FIG. A base bias value 29649 corresponding to the maximum effective area number 7 is extracted from the correspondence relationship with the bias value, and the base bias value 29649 is multiplied by each offset value of each class shown in FIG. Each offset value of each corresponding class is generated.

  The offset value of each class may be set directly so as to increase monotonously as the class increases, but in this embodiment, the offset value of each class is set to the offset coefficient and the base bias value as described above. Since the base bias value can be changed according to the histogram of the input image, even if the default offset value of each class is the same, depending on the type of the input image such as night view or darkness, etc. By appropriately setting the base bias value, the offset value of each class can be appropriately set according to the type of the input image. For this reason, by setting the offset value of each class by multiplying the offset coefficient by the base bias value in this way, the imaging apparatus CA and the image processing unit 3 of the present embodiment appropriately assign the gradations as a whole. The glare sensation peculiar to the histogram equalization method can be more appropriately suppressed.

  The cumulative histogram generation unit 325 generates a cumulative histogram from the histogram after adding the offset value generated by the offset processing unit 324. This cumulative histogram is a statistical graph in which, for each class, the sum of each frequency included in all classes up to the class is associated as a cumulative value with respect to the histogram for which the cumulative histogram is created. In the cumulative histogram, the horizontal axis is the class, and the vertical axis is the cumulative value. The cumulative histogram Mj can be expressed as Mj = Σmk (Σ is the sum of k = 1 to j) when the histogram is mk. The cumulative histogram generation unit 325 outputs the generated cumulative histogram to the normalization processing unit 326.

  The normalization processing unit 326 generates compression characteristics by normalizing the cumulative histogram generated by the cumulative histogram generation unit 325 to a range of pixel values that can be taken by the compressed image. That is, the normalized cumulative histogram becomes the compression characteristic. The range of pixel values that can be taken by the compressed image is determined according to a device that uses the compressed image, for example, a display device, a printing device, a communication device, and the like. For example, the range of pixel values that can be taken by the compressed image is determined by the dynamic range of the display device when the compressed image is displayed on the display device. The normalization processing unit 326 outputs the generated compression characteristics to the compressed image generation unit 327.

  The compressed image generation unit 327 generates a compressed image having a dynamic range smaller than that of the input image by using the compression characteristics generated by the normalization processing unit 326. More specifically, the horizontal axis of the cumulative histogram after normalization as the compression characteristic is the pixel value of the input image (for example, input luminance value, original luminance value), and the vertical axis is the pixel after dynamic range compression. Value (for example, output luminance value, compressed luminance value). The compressed image generation unit 327 obtains a compressed pixel value by converting the pixel value of each pixel of the input image with the compression characteristic, and generates a compressed image. The compressed image generation unit 327 outputs the compressed image to the luminance / chromaticity processing unit 33.

  Next, the operation of this embodiment will be described. FIG. 5 is a flowchart showing the operation of the dynamic range compression processing unit shown in FIG. FIG. 6 is a diagram for explaining the processing result generated in each processing step of the operation shown in FIG. 5 when the histogram of the input image has two peaks. FIG. 7 is a diagram for explaining the processing result generated in each processing step of the operation shown in FIG. 5 when the input image is a night view. FIG. 8 is a diagram for explaining the processing result generated in each processing step of the operation shown in FIG. 5 when the input image is dark. In each of FIGS. 6 to 8, FIG. (A) shows the histogram of the input image, FIG. (B) shows the histogram after the enlargement process, and FIG. (C) shows the histogram after the clip process. (D) shows a histogram after the offset processing, and (E) shows a cumulative histogram normalized with respect to the histogram after the offset processing (ie, compression characteristics). 6A to 8D, the horizontal axis is the class, the vertical axis is the frequency, and in FIGS. 6E to 8E, the horizontal axis is It is a class, and a vertical axis is a cumulative value. FIG. 9 is a diagram for explaining a processing result generated in each processing step of a dynamic range compression technique that performs conventional histogram equalization when a histogram of an input image is not biased. FIG. 10 is a diagram for explaining a processing result generated in each processing step of a dynamic range compression technique that performs conventional histogram equalization when a histogram of an input image is biased. 9A and 9B, FIG. 9A shows a histogram of an input image in which the horizontal axis is class and the vertical axis is frequency, and FIG. The axis shows the cumulative histogram with cumulative values, and FIG. (C) shows the normalized cumulative histogram (ie, compression characteristics). Since the figure (C) is used as a compression characteristic, the horizontal axis represents the pixel value of the input image (for example, input luminance value, original luminance value), and the vertical axis represents the pixel value after dynamic range compression (for example, Output luminance value, compressed luminance value).

  When a power switch (not shown) is turned on, the imaging apparatus CA photoelectrically converts the optical image of the subject formed on the light receiving surface of the imaging sensor 2 by the imaging optical system 1 using the imaging sensor 2. The photoelectrically converted signal is subjected to image processing by the image processing unit 3 and is output to an unillustrated display as an image signal conforming to the NTSC standard. An image of the subject is displayed on the display (not shown).

  In the image processing of the image processing unit 3, the dynamic range compression processing is performed as follows. In FIG. 5, when a signal is input from the sensor output processing unit 31, the histogram analysis unit 321 generates a histogram of pixel values of each pixel using the processing result processed by the sensor output processing unit 31, The generated histogram is analyzed, and in the present embodiment, in the generated histogram, the minimum class having the frequency (minimum effective area class), the maximum class having the frequency (maximum effective area class), and the class having the frequency. The range (effective area range) is detected. Then, the histogram analysis unit 321 outputs the generated histogram to the enlargement processing unit 322, and outputs the effective area minimum class, the effective area maximum class, and the effective area range of the analysis result to the arithmetic processing unit 35 (S1). In the processing S1, such histogram generation processing and histogram analysis processing are performed, the generated histogram is notified to the enlargement processing unit 322, and the analysis result is notified to the arithmetic processing unit 35.

  Next, when a histogram is input from the histogram analysis unit 321 in the process S2, the enlargement processing unit 322, in this embodiment, the enlargement processing unit 322 has the maximum frequency in the histogram generated by the histogram analysis unit 321. The class (effective area maximum class input from the arithmetic processing unit 35) is compared with a predetermined threshold value, and the magnitude of both is determined. As a result of the comparison determination, when the maximum class (the effective area maximum class) is smaller than the predetermined threshold, the enlargement processing unit 322 determines that the frequency including the predetermined threshold (dummy frequency) is set in advance. ), The histogram generated by the histogram analysis unit 321 is expanded in the class direction. On the other hand, as a result of the comparison determination, when the maximum class (the effective area maximum class) is equal to or greater than the predetermined threshold, the expansion processing unit 322 leaves the histogram as it is without performing the expansion process. In the process S2, such an enlargement process is performed, and the processed histogram is notified to the clip processing unit 323. Is notified to the clip processing unit 323.

  Next, when the processed histogram is input from the enlargement processing unit 322 in the process S3, the clip processing unit 323, in this embodiment, for each class with respect to the histogram processed by the enlargement processing unit 322, The frequency of the class is compared with the limit value of the class, and the magnitude of both is determined. As a result of the comparison determination, when the frequency of the class exceeds the limit value of the class, the clip processing unit 323 replaces the frequency of the class with the limit value of the class. On the other hand, as a result of the comparison determination, when the frequency of the class is equal to or lower than the limit value of the class, the clip processing unit 323 keeps the frequency of the class as it is. In the process S3, such clip processing is performed, and the processed histogram is notified to the offset processing unit 324.

  Next, in process S4, when the processed histogram is input from the clip processing unit 323, in the present embodiment, the offset processing unit 324 starts from the lowest class having the frequency in the histogram processed by the clip processing unit 323. For each class up to the maximum class having the frequency, an offset value of each class is generated by multiplying the base bias value from the arithmetic processing unit 35 by the offset coefficient of each class, and each generated class has an offset value. The offset value is added to the frequency of each class from the minimum class to the maximum class. In the process S4, such an offset process is performed, and the processed histogram is notified to the cumulative histogram generation unit 325.

  In parallel with each processing from processing S2 to processing S4, the arithmetic processing unit 35 generates setting data including at least information of the effective area maximum class based on the analysis result input from the histogram analysis unit 321. Based on the analysis result, the base bias value is generated and output as setting data to the offset processing unit 324.

  Next, in process S5, when a processed histogram is input from the offset processing unit 324, the cumulative histogram generation unit 325 generates a cumulative histogram from the histogram after the offset value addition generated by the offset processing unit 324. In the process S5, such a cumulative histogram generation process is performed, and the generated cumulative histogram is notified to the normalization processing unit 326.

  Next, when a cumulative histogram is input from the cumulative histogram generation unit 325 in the process S6, the normalization processing unit 326 normalizes the cumulative histogram generated by the cumulative histogram generation unit 325 within a range of pixel values that can be taken by the compressed image. To generate a compression characteristic. In the process S6, such a normalization process is performed, a compression characteristic is generated, and the normalized cumulative histogram is notified to the compressed image generation unit 327 as a compression characteristic.

  Next, when the compression characteristic is input from the normalization processing unit 326 in the process S7, the compressed image generation unit 327 uses the compression characteristic generated by the normalization processing unit 326, thereby increasing the dynamic range than the input image. A small compressed image is generated. In the process S 7, such a compressed image generation process is performed to generate a compressed image, and the generated compressed image is output to the luminance chromaticity processing unit 33 as an output of the DRC processing unit 32.

  As described above, the imaging apparatus CA and the image processing unit 3 in the present embodiment include the offset processing unit 324, and performs offset processing that adds an offset value that monotonously increases as the class increases to the histogram based on the input image. Generate compression characteristics. For this reason, the imaging apparatus CA and the image processing unit 3 according to the present embodiment perform gradation allocation throughout, can suppress the glare characteristic peculiar to the histogram equalization method, and more appropriately the dynamic range. Can be compressed.

  In addition, the imaging apparatus CA and the image processing unit 3 in the present embodiment include an enlargement processing unit 322, and when the histogram based on the input image is biased, the enlargement processing unit 322 enlarges the histogram in the class direction. Accordingly, the maximum class having the frequency in the histogram based on the input image becomes a larger class by the enlargement processing unit 322, and the frequency is given by the offset processing unit 324 to the class that originally did not have the frequency. For this reason, the imaging apparatus CA and the image processing unit 3 according to the present embodiment have a wider range of gradation assignments even in the case of a histogram in which the frequencies are concentrated in a predetermined range of classes and the frequencies are unevenly distributed. The glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be compressed more appropriately.

  In addition, the imaging apparatus CA and the image processing unit 3 in this embodiment include a clip processing unit 323, and the clip processing unit 323 limits the frequency to a predetermined limit value with respect to the histogram based on the input image. Therefore, even in the case of a histogram in which the frequencies are unevenly distributed, the frequency is limited to the predetermined limit value by the clip processing unit 323, so that the degree of the frequency bias with respect to the whole is reduced. For this reason, the imaging apparatus CA and the image processing unit 3 according to the present embodiment are uniform in tone allocation as a whole, can suppress the glare characteristic peculiar to the histogram equalization method, and more appropriately compress the dynamic range. can do.

  Further, in the imaging device CA and the image processing unit 3 in the present embodiment, the limit value of the clip processing is different for each class. For this reason, the imaging apparatus CA and the image processing unit 3 according to the present embodiment can adjust the limit value for each class, and therefore can adjust the gradation allocation to be uniformized as a whole. Therefore, in the imaging device CA and the image processing unit 3 in the present embodiment, it is possible to further suppress the glare feeling peculiar to the histogram equalization method, and to more appropriately compress the dynamic range.

  Here, in order to describe the operation and effect of the imaging device CA and the image processing unit 3 in the present embodiment more specifically, the generation of compression characteristics for three types of input images will be described below.

  First, as shown in FIG. 6 (A), the histogram of the input image has a partial range of a relatively dark class (small class) and a partial range of a relatively bright class (large class). In the case where each is a histogram of a lid mountain having a frequency peak, in process S2, the maximum class having the frequency (maximum effective area class) in the histogram of the lid mountain is as shown in FIG. 6B. Since the predetermined threshold value (indicated by ▲ in FIG. 6B) is exceeded, no expansion in the class direction is performed. Next, in process S3, as shown in FIG. 6 (C), the frequency of each class in the lid mountain histogram is the limit value of each class (indicated by a broken line in FIG. 6 (C)). Since it does not exceed, clip processing is not performed. Next, in process S4, an offset process is performed, and as shown in FIG. 6D, an offset value for each class is added to the frequency of each class in the histogram of the top mountain. For this reason, in the original histogram of the top mountain, the frequency is increased as the class increases, and the frequency peak in a part of a relatively bright class (large class) becomes higher. In process S5, a cumulative histogram is generated from the offset-processed peak mountain histogram, and in process S6, this is normalized. As shown in FIG. 6E, the normalized cumulative histogram generated in this way, that is, the compression characteristic, has a portion that rises relatively steeply as the class increases, and increases substantially linearly as the class increases. Profile.

  Second, when the input image is a night view, as shown in FIG. 7A, the histogram of the input image has a class having a frequency in a relatively wide range, but a relatively dark class (small class). There is a frequency peak in a part of the range, the frequency distribution is biased. In such a night scene histogram, in processing S2, the maximum class (effective area maximum class) having the frequency in the night scene histogram is a predetermined threshold (FIG. 7B) as shown in FIG. No expansion in the class direction is performed. Next, in the process S3, as shown in FIG. 7C, the frequency of each class in the histogram of the night scene is the limit value of each class (in FIG. 7C). Therefore, the clip processing is performed in this portion (the clip processing is not performed in other portions where the frequency of the class does not exceed the limit value of the class). As a result of the peak cut in this way, the degree of frequency deviation relative to the whole is reduced. Next, in the process S4, an offset process is performed, and as shown in FIG. 7D, an offset value for each class is added to the frequency of each class in the clipped night scene histogram. For this reason, in the histogram in which the frequency of the dark portion is cut by the limit value by the clipping process, the frequency is increased as the class increases, and the frequency of the relatively bright class becomes higher and emphasized. In step S5, a cumulative histogram is generated from the histogram of the night scene subjected to the offset processing. In step S6, this is normalized. As shown in FIG. 7E, the normalized cumulative histogram generated in this way, that is, the compression characteristic, has fewer parts that rise relatively steeply as the class increases, and is substantially linear as the class increases. Increased profile.

  Third, when the input image is dark, as shown in FIG. 8 (A), the histogram of the input image has frequencies only in a part of a relatively dark class (small class). Has a frequency peak in the range. In such a dark histogram with a biased frequency distribution, in process S2, the maximum class having the frequencies in this dark histogram (effective area maximum class) is a predetermined threshold value (as shown in FIG. 8B). Since it does not exceed (indicated by ▲ in FIG. 8B), enlargement processing is performed, and this dark histogram is enlarged in the class direction. As a result, the effective area range is expanded, and as a result, the entire range is expanded, so that the degree of frequency bias relative to the entire range is reduced. Next, in the process S3, as shown in FIG. 8C, the frequency of each class in the darkness histogram that has been subjected to the enlargement processing is such that the peak frequency in the dark class range is the limit value (FIG. 8). (C) (indicated by a broken line), the clip processing is performed in this portion (the clip processing is not performed in other portions where the class frequency does not exceed the limit value of the class). . As a result of the peak cut in this way, the degree of frequency deviation relative to the whole is further reduced. Next, in process S4, offset processing is performed, and as shown in FIG. 8D, the offset value for each class is added to the frequency of each class in the clipped darkness histogram. For this reason, in the histogram in which the frequency of the dark portion is cut by the limit value by the clipping process, the frequency is increased as the class increases, and the frequency of the relatively bright class becomes higher and emphasized. In process S5, a cumulative histogram is generated from the dark histogram subjected to the offset process, and in process S6, this is normalized. As shown in FIG. 8 (E), the normalized cumulative histogram generated in this way, that is, the compression characteristic, has fewer portions that rise relatively steeply as the class increases, and is substantially linear as the class increases. Increased profile.

  As described above, the imaging apparatus CA and the image processing unit 3 according to the present embodiment have a profile that increases substantially linearly as the class increases even for an input image having a mountain histogram, a night scene input image, and a dark input image. Therefore, gradation assignment is performed over the whole, and the glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be compressed more appropriately.

  As a comparative example of the compression characteristics of the present embodiment, the compression characteristics of the comparative example will be described. The compression characteristic of this comparative example is generated as follows. First, a histogram of the input image is generated, then a cumulative histogram is generated for this histogram, and then this cumulative histogram is normalized to a range of pixel values that can be taken by the compressed image. Thereby, the compression characteristic of the comparative example is generated.

  In such a compression characteristic generation method of the comparative example, as shown in FIG. 9A, the histogram of the input image is uniformly distributed from a relatively dark class (small class) to a bright class (large class). 9B, as shown in FIG. 9B, the cumulative histogram generated from the histogram shown in FIG. 9A has a profile that increases substantially linearly. Therefore, the cumulative histogram obtained by normalizing this, that is, the compression characteristic, is a characteristic curve having a profile that increases substantially linearly as shown in FIGS. 9C and 9D, respectively. For this reason, gradations are assigned substantially uniformly throughout, and a compressed image that is easy to view can be obtained.

  However, in the case where the input image is dark or night scene, the histogram of such an input image has a frequency class biased to a partial range (dark class range) as shown in FIG. (In the case of darkness), or a class having a frequency in a relatively wide range, but a class having a large frequency is biased to a certain range (in the dark class range) (in the case of a night view). As shown in FIG. 10B, the cumulative histogram generated from such a histogram having a frequency-biased profile in a relatively dark class rises relatively steeply as the class increases, and then gradually increases. Profile. Therefore, as shown in FIGS. 10C and 10D, the cumulative histogram obtained by normalizing this, that is, the compression characteristic, rises relatively steeply as the class increases, and thereafter, as in the cumulative histogram. The characteristic curve has a gradually increasing profile, and does not become a characteristic curve having a substantially linearly increasing profile. For this reason, gradation is preferentially assigned to the portion that rises relatively steeply as the class increases (the portion circled with a broken line in FIG. 10D), and gradation assignment is locally performed. Become. As a result, the compressed image has a glare that is characteristic of the histogram equalization method.

  On the other hand, in the imaging apparatus CA and the image processing unit 3 according to the present embodiment, as shown in FIGS. 6E, 7E, and 8E, the cumulative histogram is compared with FIG. 10B. The profile increases more linearly as the class increases. Therefore, the compression characteristics obtained by normalizing this cumulative histogram also increase more linearly as the class increases in the imaging apparatus CA and the image processing unit 3 in this embodiment, as shown in FIG. It becomes a characteristic curve with a profile. As a result, in the imaging apparatus CA and the image processing unit 3 according to the present embodiment, even in the case of a histogram in which the frequencies are concentrated in a predetermined range of classes and the frequencies are unevenly distributed, the gradation allocation is over a wider range. The glare feeling peculiar to the histogram equalization method can be suppressed, and the dynamic range can be compressed more appropriately.

  In the above description, the DRC processing unit 32 includes a histogram analysis unit 321, an enlargement processing unit 322, a clip processing unit 323, an offset processing unit 324, a cumulative histogram generation unit 325, a normalization processing unit 326, and a compressed image generation unit 327. However, for example, the enlargement processing unit 322 is omitted, and the DRC processing unit 32 includes a histogram analysis unit 321, a clip processing unit 323, an offset processing unit 324, a cumulative histogram generation unit 325, a normalization processing unit 326, and a compressed image generation unit. 327 may be provided. In such a case, the clip processing unit 323 performs clip processing for limiting the frequency to a predetermined limit value for the histogram generated by the histogram analysis unit 321. For example, the clip processing unit 323 is omitted, and the DRC processing unit 32 includes a histogram analysis unit 321, an enlargement processing unit 322, an offset processing unit 324, a cumulative histogram generation unit 325, a normalization processing unit 326, and a compressed image generation unit 327. It may be provided. In such a case, the offset processing unit 324 applies the class of each class from the lowest class having the frequency to the maximum class given the predetermined frequency in the histogram enlarged by the enlargement processing unit 322. The offset value of each class that increases monotonously with the increase is generated, and the generated offset value is added to the frequency of each class from the minimum class to the maximum class. Further, for example, the enlargement processing unit 322 and the clip processing unit 323 are omitted, and the DRC processing unit 32 includes a histogram analysis unit 321, an offset processing unit 324, a cumulative histogram generation unit 325, a normalization processing unit 326, and a compressed image generation unit 327. It may be provided. In such a case, the offset processing unit 324 applies a class for each class from the lowest class having the frequency to the maximum class given the predetermined frequency in the histogram generated by the histogram analysis unit 321. The offset value of each class that increases monotonously with the increase is generated, and the generated offset value is added to the frequency of each class from the minimum class to the maximum class.

  Further, in the above-described embodiment, the imaging device CA and the image processing unit 3 have a plurality of offset values monotonously increasing as the class increases in the offset processing unit 324, as indicated by broken lines in FIG. You may further provide the multiple pattern production | generation process part 3241 produced | generated by a pattern, respectively. With this configuration, it is possible to generate a compressed image that has been offset with a plurality of patterns, so various compressed images can be generated with each compression characteristic according to the offset value of each pattern for one input image. It is possible to generate and select an optimal compressed image from these various compressed images.

  In order to express the present invention, the present invention has been properly and fully described through the embodiments with reference to the drawings. However, those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that this is possible. Therefore, unless the modifications or improvements implemented by those skilled in the art are at a level that departs from the scope of the claims recited in the claims, the modifications or improvements are not covered by the claims. To be construed as inclusive.

CA imaging device 1 imaging optical system 2 imaging sensor 3 image processing unit 31 sensor output processing unit 32 dynamic range compression processing unit 321 histogram analysis unit 322 enlargement processing unit 323 clip processing unit 324 offset processing unit 325 cumulative histogram generation unit 326 normalization processing 327 Compressed image generation unit

Claims (8)

A histogram generator that generates a histogram of pixel values of each pixel in the input image;
For each class from the lowest class with frequency to the highest class with frequency in the histogram generated by the histogram generation unit, an offset value for each class that increases monotonically as the class increases is generated, and the generation An offset processing unit that adds the offset value of each class to the frequency of each class from the minimum class to the maximum class,
A compression characteristic generation unit that generates a compression characteristic for compressing the input image based on the histogram after the offset value addition generated by the offset processing unit;
An image processing apparatus comprising: a compressed image generation unit that generates a compressed image having a smaller dynamic range than the input image by using the compression characteristic generated by the compression characteristic generation unit. An enlargement processing unit for enlarging the histogram generated by the histogram generation unit in the class direction by giving a predetermined frequency to a class larger than the maximum class having the frequency in the histogram generated by the histogram generation unit;
The offset processing unit monotonously increases as the class increases for each class from the lowest class having the frequency to the highest class given the predetermined frequency in the histogram enlarged by the enlargement processing unit. The image according to claim 1, wherein an offset value for each class is generated, and the generated offset value for each class is added to the frequency of each class from the minimum class to the maximum class. Processing equipment. A clip processing unit that limits the frequency to a predetermined limit value with respect to the histogram generated by the histogram generation unit,
The offset processing unit, for each class from the lowest class having the frequency to the highest class having the frequency in the histogram processed by the clip processing unit, the offset value of each class monotonously increasing as the class increases The image processing apparatus according to claim 1, further comprising: generating an offset value of each class and adding the generated offset value of each class to the frequency of each class from the minimum class to the maximum class. A clip processing unit that limits the frequency to a predetermined limit value for the histogram expanded by the expansion processing unit,
The offset processing unit, for each class from the lowest class having the frequency to the highest class having the frequency in the histogram processed by the clip processing unit, the offset value of each class monotonously increasing as the class increases The image processing apparatus according to claim 2, further comprising: generating an offset value of each class, and adding the generated offset value of each class to the frequency of each class from the minimum class to the maximum class. The image processing apparatus according to claim 3, wherein the predetermined limit value is different for each class. 6. The multi-pattern generation processing unit that causes the offset processing unit to generate offset values of each class that monotonously increase as the class increases in a plurality of patterns, respectively. The image processing apparatus according to claim 1. A histogram generation step of generating a histogram of pixel values of each pixel in the input image;
In the histogram generated in the histogram generation step, for each class from the lowest class having the frequency to the highest class having the frequency, an offset value of each class monotonically increasing as the class increases is generated, and the generation Offset processing step of adding the offset value of each class to the frequency of each class from the minimum class to the maximum class,
A compression characteristic generation step for generating a compression characteristic for compressing the input image based on the histogram after the offset value addition generated in the offset processing step;
An image processing method comprising: a compressed image generation step of generating a compressed image having a smaller dynamic range than the input image by using the compression characteristic generated in the compression characteristic generation step. An imaging unit that photoelectrically converts a subject light image to generate an image signal;
An image processing unit that performs predetermined image processing on the image signal generated by the imaging unit and generates an image;
The image processing unit includes the image processing apparatus according to any one of claims 1 to 6.

Images