JP2014006698A - Image processing device, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize gradation compression with a contrast maintained for a wide gradation dynamic-range infrared image, regardless of the size of the area of an object to be focused.SOLUTION: A layer elimination part 11 measures distribution of appearance frequency of a pixel value in an input image, and in the case of making the measured distribution a curve, defines, as a cluster, a pixel value between two pixel values whose appearance frequency is the minimal value compared to the surrounding pixel values, the two pixel values sandwiching a pixel value whose appearance frequency is the maximal value compared to the surrounding pixel values, and extracts all clusters. For each cluster, the layer elimination part 11 defines a weighting determined for a central pixel value included in the cluster as the weighting thereof and assigns a new pixel value to each pixel value included in the cluster so that the assigned gradation increases with the weight of the cluster. A gradation conversion part 15 uses the new pixel value assigned by the layer elimination part 11 to convert the input image, thus generating an output image.

Description

  The present invention relates to a gradation compression technique for converting an image output from an infrared sensor into a limited dynamic range.

  In recent years, due to the improvement in performance of infrared sensors, the dynamic range of infrared images that can be captured has been expanded. For example, there is one having a dynamic range of about 16 bits. Infrared radiation of a hot object is high and infrared radiation of a low temperature object is low.In the image output from the infrared sensor, a high temperature object is output with a high pixel value and a low temperature object is output with a low pixel value. The The wide dynamic range of the infrared sensor means that images from a low temperature subject to a high temperature subject can be captured at a time. For example, even when shooting a jet in winter, it is useful because both the low temperature background in winter and the engine portion of the high temperature jet are visible.

  On the other hand, there is a limit to the luminance gradation that can be handled by a device that displays an image, and 8 bits are still mainstream at present. For this reason, even if an image having a wide dynamic range is captured, there is currently no device that can display the image with its contrast maintained.

Therefore, there is a need for a gradation compression technique for generating an image adapted to the dynamic range of a display device by compressing the gradation dynamic range of an image having a wide dynamic range to be narrowed.
The conventional gradation compression technique is realized by redistributing gradations of an image having a wide dynamic range to a narrow dynamic range. However, for example, if the gradation of an image with a wide dynamic range is simply redistributed evenly to a narrow dynamic range by linear conversion, the luminance change of the entire image will be reduced, and the image will be converted into an invisible image with reduced contrast. It will end up. Therefore, several gradation compression techniques that can suppress a decrease in contrast have been proposed.

Japanese Patent Application Laid-Open No. 2005-228561 describes that gradation redistribution rules are adaptively determined based on a cumulative frequency distribution of luminance of an input wide dynamic range image so that a histogram of the cumulative frequency distribution becomes flat. is there. This gradation compression technique assumes that a subject with a large area ratio in the image is an important subject, and has a luminance value with a high frequency distribution (a large area in the image) and a luminance value in the vicinity thereof. A gradation conversion curve is determined so as to distribute as many gradations as possible, and at least a decrease in contrast in an important subject is suppressed.
However, with the method described in Patent Document 1, it is difficult to obtain satisfactory results in all situations when applied to infrared images. For example, when an important subject is small or there are a plurality of important subjects in an image, sufficient gradation may not be distributed to the important subject.

Japanese Patent Application Laid-Open No. H10-228561 describes that high-frequency components are emphasized in order to maintain the contrast of the contour in the image either before or after gradation conversion. In this gradation compression technique, the contrast lost by gradation compression (or lost by gradation conversion) is estimated from an image, and the corresponding amount is compensated by a high frequency enhancement filter.
The method described in Patent Document 2 has an advantage that it does not depend on other subjects as in the method described in Patent Document 1. However, the high-frequency emphasis filter has problems such as overshooting in the contour portion of the subject and emphasizing noise in a portion where the gradation is flat. In addition, when the subject is small, it is not always possible to obtain an easy-to-see image, for example, the appearance changes due to the filter characteristics, affecting the shape and luminance distribution of the subject.

JP-A-9-331469 JP 2000-115534 A

  The methods described in Patent Documents 1 and 2 are frequently used because they can generate a natural image in an image captured by a visible light sensor. However, since the infrared image captured by the infrared sensor is often very small in size with respect to the background imaged by the infrared sensor, the method described in Patent Documents 1 and 2 performs gradation compression to an image that is easy to see. There are many things that cannot be done.

For example, when a general urban area is imaged with an infrared sensor at night, roads, buildings, ground, vegetation, etc. are temperatures close to the outside temperature and low in brightness, and occupy a large area in the image. Although the brightness is high because of high temperature, the area occupied by the image is small. Similarly, the temperature of only the exhaust nozzle portion is high in the vehicle, and the area occupied in the image is small.
Also, for example, when searching for a flying object or an aircraft by monitoring the sky with an infrared camera, the background of the image with low temperature and low brightness such as the sky and clouds occupies most of the image, and the radiation of the flying object and aircraft The exhaust gas temperature is high and the luminance is high, but the area occupied in the image is small. The temperature of the flying body and the aircraft body is slightly higher than the background such as the sky, but it is significantly lower than the exhaust gas, and the area occupied by the image is small.
Street lights, vehicle exhaust nozzles, flying objects and aircraft are called targets. The target may be as large as several pixels to one pixel.

  On the other hand, when the method described in Patent Document 1 is employed, the target pixel value cumulative frequency is small because the target area is small, and the target tone redistribution amount in tone conversion is small. The contrast is lowered. In particular, the aircraft body among the targets has a small gradation difference in luminance with the background and a small area, so that the gradation difference from the background after gradation compression becomes very small, making it difficult to distinguish from the background. In addition, since non-linear tone compression is performed, for example, a pattern indicating a road, a building, a vehicle, an aircraft, or the like changes, and it is difficult to identify the type or the like.

  Further, when the method described in Patent Document 2 is adopted, the target size is small, so that the shape and brightness change due to the influence of the high-frequency emphasis filter, and cannot be recognized as the same target. In addition, since non-linear tone compression is performed, for example, a pattern indicating a road, a building, a vehicle, an aircraft, or the like changes, and it is difficult to identify the type or the like.

  An object of the present invention is to realize gradation compression that maintains contrast for an infrared image having a wide gradation dynamic range regardless of the size of the subject to be noted.

An image processing apparatus according to the present invention includes:
An image processing device that generates an output image with a reduced dynamic range of gradation of an input image,
A distribution measurement unit that measures the distribution of the appearance frequency of pixel values in the input image;
When the distribution measured by the distribution measuring unit is analyzed by clustering the distribution of the appearance frequency measured by the distribution measuring unit, the pixel value at which the appearance frequency becomes a maximum value higher than the surrounding pixel values. A cluster extraction unit that extracts a plurality of clusters included in the distribution, with a plurality of pixel values between two pixel values having a minimum value lower than the surrounding pixel values as one cluster. ,
A weight storage unit storing weights for each pixel value;
For each cluster extracted by the cluster extraction unit, the weight stored by the weight storage unit with respect to a predetermined pixel value included in the cluster is set as the weight of the cluster, and the higher the weight assigned to the cluster, the larger the gradation assigned. As described above, a gradation assigning unit that assigns a new pixel value to each pixel value included in each cluster;
And a tone conversion unit that converts the input image using the new pixel value assigned by the tone assigning unit to generate an output image.

  In the image processing apparatus according to the present invention, by assigning gradations to the extracted clusters, useless distribution to unused gradations can be suppressed. Furthermore, by controlling the distribution of gradation according to the weight, it is possible to maintain the contrast of the subject that is particularly noted by the user.

1 is a functional block diagram of an image processing apparatus 10 according to a first embodiment. 3 is a flowchart showing the operation of the image processing apparatus 10 according to the first embodiment. FIG. 3 is an explanatory diagram of processing of the image processing apparatus 10 according to the first embodiment. FIG. 4 is a functional block diagram of an image processing apparatus 10 according to a second embodiment. 1 is a hardware configuration diagram of an image processing apparatus 10 according to Embodiments 1 and 2. FIG.

Embodiment 1 FIG.
FIG. 1 is a functional block diagram of an image processing apparatus 10 according to the first embodiment.
The image processing apparatus 10 is an apparatus that performs gradation compression of the input image 1 and generates an output image 6 in which the gradation dynamic range is narrowed. Here, a description will be given assuming that the 16-bit gradation input image 1 is gradation-compressed into an 8-bit gradation output image 6.

The image processing apparatus 10 includes a hierarchy reduction unit 11, a gradation conversion unit 15, and a weight storage unit 16. The hierarchy reduction unit 11 includes a distribution measurement unit 12, a cluster extraction unit 13, and a gradation allocation unit 14.
In the image processing apparatus 10, first, the hierarchy reducing unit 11 receives the input image 1 and the weight 4 stored in the weight storage unit 16 as input, and generates a conversion curve 5 adapted to the luminance distribution and the weight 4 of the input image 1. Generate. Then, the gradation conversion unit 15 converts the input image 1 based on the conversion curve 5 and generates an output image 6.
Details of each function of the image processing apparatus 10 will be described together with an operation flow of the image processing apparatus 10.

FIG. 2 is a flowchart showing the operation of the image processing apparatus 10 according to the first embodiment. FIG. 3 is an explanatory diagram of processing of the image processing apparatus 10 according to the first embodiment.
(S1: Distribution measurement process)
The distribution measuring unit 12 is an infrared image input from an infrared sensor, and receives an input image 1 that is an infrared image having a wide dynamic range of gradation (here, 16-bit gradation). The distribution measurement unit 12 measures the number of pixels of each pixel value included in the input image 1 as the appearance frequency of the pixel value by the processing device, and outputs a frequency distribution 2 indicating the appearance frequency for each pixel value.

(S2: Cluster extraction process)
The cluster extraction unit 13 generates a histogram curve obtained by curving the frequency distribution 2 by the processing device (see FIG. 3A).
The histogram curve is generated by making the frequency distribution 2 into a histogram with the horizontal axis representing the pixel value and the vertical axis representing the appearance frequency, and connecting the top points of the bars indicating the appearance frequency of each pixel value by spline interpolation or the like.

  The cluster extraction unit 13 extracts all the clusters from the histogram curve by using the processing apparatus as a cluster of pixel values, with the peaks of the histogram curve as a group (see FIG. 3B). Note that the peaks of the histogram curve are two peaks in the histogram curve between which the appearance frequency is a high maximum value compared to the surrounding pixel values and the appearance frequency is a low minimum value compared to the surrounding pixel values. The part between pixel values.

For each extracted cluster, the cluster extraction unit 13 outputs the pixel value included in the cluster, the central pixel value of the cluster, and the pixel value width of the cluster as cluster information 3. Note that the pixel value width of a cluster is the number of pixel values included in the cluster. For the clusters c ₁ , c ₂ , c ₃ , and c ₄ in FIG. 3C, x ₁ , x ₂ , This is information indicated by x ₃ and x ₄ .

  Note that the peaks of the histogram curve often correspond to the type of subject in the input image 1 in the infrared image. For example, it is assumed that the sky, clouds, and an aircraft are captured as subjects of the input image 1. The sky and clouds are the background and occupy most of the area of the input image 1, but the temperature of the clouds is a little higher than the sky, so if the frequency distribution 2 is a histogram curve, the sky and the clouds Can be done separately. The aircraft has a smaller area than the background, but the temperature is higher than the background and consists of two temperatures, the fuselage and the exhaust. Therefore, the histogram curve has two more mountains apart from the sky and clouds. Since each subject has a width of pixel values, each mountain in the histogram curve is a mountain-like curve having a width. The cluster extraction unit 13 searches for this mountain-shaped curve from the frequency distribution 2, identifies the center pixel value and its width for each subject, and outputs this as cluster information 3.

(S3: gradation assignment process)
The gradation assigning unit 14 assigns a gradation to each cluster based on the central pixel value and the pixel value width included in the cluster information 3 and the weight 4 for each pixel value stored in the weight storage unit 16, and the conversion curve 5 Is generated by the processing device.
Specifically, the gradation assigning unit 14 increases the distribution of the pixel value width of the cluster in the output pixel value width as the weight 4 corresponding to the central pixel value of each cluster increases, A new pixel value is assigned to a pixel value included in each cluster so that the distribution of the pixel value width of the cluster in the pixel value width (8 bits in this case) of the output image 6 increases as the pixel value width increases. Then, the gradation assigning unit 14 converts a numerical table in which a pixel value included in each cluster is an input pixel value, and a new pixel value assigned to the input pixel value is an output pixel value for the input pixel value. 5 (see FIG. 3E).
Note that the gradation assigning unit 14 redistributes the gradation of the output image 6 to a pixel value that is not at all in the input image 1 that has not been extracted as a cluster and to a very small pixel value (often noise). Reduce unnecessary occupancy of gradations. This increases the amount of reallocation to gradations with high appearance frequency.

  The weight 4 is assigned to each pixel value included in the pixel value width (here, 16 bits) of the input image 1. For example, as shown in FIG. 3D, for the weight 4, a small value is assigned to the pixel value indicating the background, and a large value is assigned to the pixel value indicating the target.

(S4: gradation conversion process)
The gradation converting unit 15 converts each pixel value of the input image 1 as an input pixel value of the conversion curve 5 and converts each pixel value of the input image 1 into an output pixel value corresponding to the input pixel value, thereby obtaining a 16-bit gradation. The input image 1 is converted into 8-bit gradation by the processing device to generate and output the output image 6.

As described above, in the image processing apparatus 10 according to this embodiment, in the gradation compression from the input image 1 to the output image 6, the distribution of the unused gradations that are not used is suppressed, and the overall contrast is reduced. Will improve. Furthermore, by giving the effect of maintaining the contrast of the subject of interest to the user according to the weight 4, the user's ability to identify the subject is improved. At the same time, the amount of transmitted data can be reduced, and the system scale and cost can be reduced.
For example, as an input image 1, an infrared image is input, with the sky and clouds as the background and the aircraft and its exhaust as a target (see FIG. 3A). Further, as the weight 4, the gradation weight corresponding to the background temperature is set small, and the gradation weight corresponding to the target temperature is set large (see FIG. 3D). In this case, since the target area occupied in the input image 1 is generally smaller than the area occupied in the background, in the conventional method, the background contrast is high and easy to identify. Is difficult to identify (see FIG. 3B). However, in the image processing apparatus 10 according to the first embodiment, it is possible to perform gradation compression of an image having a wide dynamic range into an image having a narrow dynamic range while maintaining both the background contrast and the aircraft contrast. Therefore, an image that can be easily identified by the user can be supplied through the display device (see FIG. 3E). In addition, since linear gradation compression is possible for each subject, a pattern such as a road, a building, a vehicle, or an aircraft can be maintained, and the type or the like can be identified.

  In the above description, the luminance of the input image 1 is assumed to have a 16-bit gradation, and the pixel value of the output image 6 is assumed to have an 8-bit gradation. However, as long as the gradation of the output image 6 is narrower than the gradation of the input image 1, the gradation can be similarly compressed regardless of the gradation.

Embodiment 2. FIG.
In the second embodiment, an image processing apparatus 10 to which a weight 4 generation function is added will be described.
In the second embodiment, the image processing apparatus 10 will be described with a focus on differences from the image processing apparatus 10 according to the first embodiment.

FIG. 4 is a functional block diagram of the image processing apparatus 10 according to the second embodiment.
The image processing apparatus 10 includes a target detection unit 17 and a weight determination unit 18 in addition to the functions of the image processing apparatus 10 illustrated in FIG.

The target detection unit 17 detects a target from the input image 1 based on the shape and outputs the target pixel value, area (number of pixels), and pixel value width as target information 7. The weight determination unit 18 receives the target information 7 and outputs a weight 8 in which the weight of the pixel value where the target exists is set large and the weight of the pixel value where the target does not exist is set small.
And the weight memory | storage part 16 memorize | stores the weight 8 which the weight determination part 18 output as the weight 4, and the hierarchy reduction part 11 uses the weight 4 set automatically in this way.

In the image processing apparatus 10 according to the first embodiment, the target pixel value needs to be known in advance, or the weight of a wide pixel value range is increased when the target pixel value is unknown in advance. There is a need.
However, the image processing apparatus 10 according to the second embodiment automatically generates the optimum weight 4 for target identification even when the target pixel value is not known in advance. Therefore, it is possible to gradation-compress an image with a wide dynamic range into an image with a narrow dynamic range while maintaining the target contrast, and it is possible to supply an image that is easy for the user to identify even through the display device.

  Note that the image processing apparatus 10 according to the above embodiment is generally applicable to an apparatus using an infrared sensor, and an infrared camera, a navigation apparatus, a background, and the like that display an image and supply it to a user. Can be used for target search devices that search for and detect targets using different infrared characteristics, gas detectors that display the distribution of substances from the infrared characteristic wavelength band, and infrared astronomical observation equipment.

FIG. 5 is a hardware configuration diagram of the image processing apparatus 10.
As shown in FIG. 5, the image processing apparatus 10 includes a CPU 911 (also referred to as a central processing unit, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a processor) that executes a program. The CPU 911 is connected to the ROM 913, the RAM 914, the LCD 901 (Liquid Crystal Display), the keyboard 902 (K / B), the communication board 915, and the magnetic disk device 920 via the bus 912, and controls these hardware devices. Instead of the magnetic disk device 920 (fixed disk device), a storage device such as an optical disk device or a memory card read / write device may be used. The magnetic disk device 920 is connected via a predetermined fixed disk interface.

  An operating system 921 (OS), a window system 922, a program group 923, and a file group 924 are stored in the magnetic disk device 920 or the ROM 913. The programs in the program group 923 are executed by the CPU 911, the operating system 921, and the window system 922.

The program group 923 includes “hierarchy reduction unit 11”, “distribution measurement unit 12”, “cluster extraction unit 13”, “tone allocation unit 14”, “tone conversion unit 15”, “weight storage” in the above description. Software, programs, and other programs that execute the functions described as “unit 16”, “target detection unit 17”, “weight determination unit 18”, and the like are stored. The program is read and executed by the CPU 911.
The file group 924 includes “input image 1”, “frequency distribution 2”, “cluster information 3”, “weight 4”, “conversion curve 5”, “output image 6”, “target information 7” in the above description. , Information such as “weight 8”, data, signal values, variable values, and parameters are stored as items of “database”. The “database” is stored in a recording medium such as a disk or a memory. Information, data, signal values, variable values, and parameters stored in a storage medium such as a disk or memory are read out to the main memory or cache memory by the CPU 911 via a read / write circuit, and extracted, searched, referenced, compared, and calculated. Used for the operation of the CPU 911 such as calculation / processing / output / printing / display. Information, data, signal values, variable values, and parameters are temporarily stored in the main memory, cache memory, and buffer memory during the operation of the CPU 911 for extraction, search, reference, comparison, calculation, calculation, processing, output, printing, and display. Is remembered.

In the above description, the arrows in the flowchart mainly indicate input / output of data and signals, and the data and signal values are recorded in a memory of the RAM 914, other recording media such as an optical disk, and an IC chip. Data and signals are transmitted online by a bus 912, signal lines, cables, other transmission media, and radio waves.
In addition, what is described as “to part” in the above description may be “to circuit”, “to device”, “to device”, “to means”, and “to function”. It may be “step”, “˜procedure”, “˜processing”. In addition, what is described as “˜device” may be “˜circuit”, “˜device”, “˜means”, “˜function”, and “˜step”, “˜procedure”, “ ~ Process ". That is, what is described as “˜unit” may be realized by firmware stored in the ROM 913. Alternatively, only software, hardware such as elements, devices, substrates, and wirings, a combination of software and hardware, or a combination of firmware may be used. Firmware and software are stored in a recording medium such as ROM 913 as a program. The program is read by the CPU 911 and executed by the CPU 911. That is, the program causes a computer or the like to function as the “˜unit” described above. Alternatively, the computer or the like is caused to execute the procedures and methods of “to part” described above.

  1 input image, 2 frequency distribution, 3 cluster information, 4 weight, 5 transformation curve, 6 output image, 7 target information, 8 weight, 10 image processing device, 11 hierarchy reduction unit, 12 distribution measurement unit, 13 cluster extraction unit, 14 gradation allocation unit, 15 gradation conversion unit, 16 weight storage unit, 17 target detection unit, 18 weight determination unit.

Claims (6)

An image processing device that generates an output image with a reduced dynamic range of gradation of an input image,
A distribution measurement unit that measures the distribution of the appearance frequency of pixel values in the input image;
When the distribution measured by the distribution measuring unit is curved, a pixel value having an appearance frequency that is a maximum value higher than the surrounding pixel values is sandwiched, and the appearance frequency is a minimum value that is lower than the surrounding pixel values. A plurality of pixel values between two pixel values as one cluster, and a cluster extraction unit that extracts a plurality of clusters included in the distribution;
A weight storage unit storing weights for each pixel value;
For each cluster extracted by the cluster extraction unit, the weight stored by the weight storage unit with respect to a predetermined pixel value included in the cluster is set as the weight of the cluster, and the higher the weight assigned to the cluster, the larger the gradation assigned. As described above, a gradation assigning unit that assigns a new pixel value to each pixel value included in each cluster;
An image processing apparatus comprising: a gradation conversion unit configured to convert the input image using the new pixel value assigned by the gradation assignment unit to generate an output image. The gradation assigning unit increases the assigned gradation as the weight is increased, and increases the assigned gradation as the number of pixel values included in the cluster increases, so that a new pixel is added to each pixel value included in each cluster. The image processing apparatus according to claim 1, wherein a value is assigned. The image processing apparatus according to claim 1, wherein the gradation assigning unit assigns a new pixel value only to a pixel value included in any one of the clusters extracted by the cluster extracting unit. The image processing apparatus further includes:
A target identification unit for detecting a predetermined target from the input image and identifying a pixel value of the detected target;
A weight determining unit that gives a heavier weight than other pixel values to a predetermined range of pixel values including the target pixel value specified by the target specifying unit;
The image processing apparatus according to claim 1, wherein the weight storage unit stores the weight determined by the weight determination unit. An image processing method for generating an output image with a reduced dynamic range of gradation of an input image,
A distribution measuring step in which the processing device measures a distribution of appearance frequencies of pixel values in the input image;
When the processing apparatus curves the distribution measured in the distribution measurement step, the appearance frequency is lower than the surrounding pixel values with the appearance frequency sandwiching a pixel value having a maximum value higher than the surrounding pixel values. A cluster extraction step of extracting a plurality of clusters included in the distribution, with a plurality of pixel values between two pixel values serving as minimum values as one cluster;
For each cluster extracted in the cluster extraction step by the processing device, the weight stored in the storage device in advance for the predetermined pixel value included in the cluster is used as the weight of the cluster, and the gradation assigned to the cluster with a higher weight A gradation assigning step for assigning a new pixel value to each pixel value included in each cluster,
An image processing method, comprising: a gradation conversion step in which the processing device converts the input image using the new pixel value allocated in the gradation allocation step to generate an output image. An image processing program for generating an output image with a reduced dynamic range of gradation of an input image,
A distribution measurement process for measuring a distribution of appearance frequencies of pixel values in the input image;
When the distribution measured by the distribution measurement process is curved, a pixel value whose appearance frequency is a maximum value higher than the surrounding pixel values is sandwiched, and the appearance frequency is a minimum value lower than the surrounding pixel values. A cluster extraction process for extracting a plurality of clusters included in the distribution, with a plurality of pixel values between two pixel values as one cluster;
For each cluster extracted in the cluster extraction process, the weight stored in advance in the storage device for the predetermined pixel value included in the cluster is used as the weight of the cluster, so that the higher the weight assigned to the cluster, the larger the gradation assigned. Gradation allocation processing for assigning a new pixel value to each pixel value included in each cluster,
An image processing program for causing a computer to execute gradation conversion processing for converting the input image and generating an output image using a new pixel value assigned in the gradation assignment processing.

Images