JP2012014628A - Image display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable each observation target object to be clearly displayed even when high luminance noises are included in an HDR image or when various observation target objects are mixed in the HDR image.SOLUTION: An image display device comprises: a luminance conversion target area setting unit 3 for classifying pixels each having luminance similar to luminance of adjacent pixels among pixels constituting an HDR image into the same group, and setting areas where the pixels belonging to the same group exist to luminance conversion target areas (1)-(N); and a luminance conversion unit 4 for removing noises existing in the luminance conversion target areas (1)-(N) and then converting the luminance of pixels existing in the luminance conversion target areas (1)-(N) into luminance in a visible range.

Description

  The present invention converts a high dynamic range image (hereinafter referred to as an “HDR image”) having a wide range of luminance into an image having a luminance range that can be visually recognized by a human and can be displayed and displayed. The present invention relates to a display device.

As an HDR image, for example, a SAR (synthetic aperture radar) image which is a radio wave image acquired using a synthetic aperture radar (a high-intensity noise in a narrow range is added to this image), a high dynamic range camera, or the like There are images taken using.
When a synthetic aperture radar is mounted on an aircraft or an artificial satellite and the data obtained by observing the ground is expressed as an image using the synthetic aperture radar, the luminance of a pixel may be tens of thousands of gradations.
However, since there are hundreds of luminance gradations that allow humans to identify differences in luminance, in order to display an HDR image that is data observed using a synthetic aperture radar, the HDR gradation is displayed. It is necessary to convert the luminance of the pixels constituting the image into luminance within the human visible range (and hundreds of luminance gradations).

Here, a method called tone mapping (hereinafter referred to as “TMO method”) is known as a method for converting the luminance of pixels constituting an HDR image into luminance within a human visible range. Yes.
Of the data observed using synthetic aperture radar, the data related to observation objects that easily reflect radio waves such as metal has high brightness, and the data related to observation objects that are difficult to reflect radio waves such as the water surface The brightness tends to be low.
For this reason, when data observed using a synthetic aperture radar is used as an HDR image, for example, a portion expressed with high luminance like a vehicle is displayed and at the same time expressed with low luminance like a paddy field. If an attempt is made to display a portion, the difference in luminance is too large, and if both luminance ranges are not devised at the same time, an unclear image is displayed.

Specifically, it is as follows.
It is assumed that the data observed from above using the synthetic aperture radar is an HDR image in which structures around the building are shown, for example, as shown in FIG.
In this case, it is assumed that the luminance distribution of the observation target in the HDR image is a luminance distribution as shown in FIG.
At this time, if the luminance of the observation object in the HDR image is linearly converted into the luminance within the visible range using the TMO method, the luminance of most of the observation objects becomes a small value as shown in FIG.
As a result, as shown in FIG. 18, the image is converted into a substantially dark image, and what is visible becomes only noise with extremely high luminance, so that an unclear image is displayed.

In Non-Patent Document 1 below, even when a conventional TMO method for optical photography is applied, the observation target is separated into various luminance ranges, as in data observed using a synthetic aperture radar. It describes that it is not suitable for visualization of recorded HDR images.
That is, in the following Non-Patent Document 1, when the TMO method for optical photography is applied, the entire screen is converted to appear white, or it is necessary for a person to determine an application parameter through trial and error. Is described.
Further, in data observed using a synthetic aperture radar, high luminance noise appears irregularly in a narrow region of the HDR image, and it is considered that it is a problem to remove this noise.

In Patent Document 1 below, when performing luminance conversion of an HDR image, the HDR image is divided into a relatively bright region and a relatively dark region, and TMO conversion that seems to be optimal for each region is performed. A method of synthesizing the converted areas is disclosed.
In the case of this method, moderate brightness conversion is possible if the HDR image clearly separates the high brightness area and the low brightness area.
However, in the case of data observed from the sky using synthetic aperture radar, for example, if there is a building on a flat ground and the roof of the building is flat, the ground is expressed with low brightness, The edge of the building is expressed with high brightness, and the flat part of the roof of the building is expressed with high brightness.
That is, since the high luminance region and the low luminance region are expressed in a mixed manner, the high luminance region and the low luminance region cannot be appropriately separated, and it is difficult to apply the TMO method.

JP 2006-345509 A (paragraph number [0006])

Lambers, M. Nies, H. Kolb, A .: “Interactive Dynamic Range Reduction for SAR Images”, Geoscience and Remote Sensing Letters, Vol.5 Issue3 pp. 507-511, 2008

Since the conventional image display apparatus is configured as described above, when high-intensity noise appears near the observation object, there is a problem that the observation object cannot be displayed in an easy-to-understand manner due to the noise. there were.
In addition, when various observation objects are mixed in the HDR image, the high-luminance area and the low-luminance area cannot be appropriately separated, and each observation object can be displayed in an easy-to-understand manner. There was a problem that could not be done.

In addition, when the difference in luminance of a plurality of observation objects mixed in the HDR image represents a difference in shape or state of the plurality of observation objects, if the difference in luminance is small, However, since it is difficult to recognize the difference in luminance, there is a problem that it is impossible to accurately grasp the difference in shape and state of a plurality of observation objects.
Furthermore, when a method of automatically determining the application parameter of the TMO method is adopted instead of a human being trial and error to determine the application parameter, not only the region in the HDR image that needs to be visualized, It is assumed that an application parameter is determined in consideration of an area where visualization is unnecessary.
For this reason, there is a problem that an optimum application parameter for luminance conversion of an area that needs to be visualized is not always determined, and an area that needs to be visualized cannot be displayed in an easily understandable manner.

The present invention has been made to solve the above-described problems, and even when high brightness noise is included in an HDR image or when various observation objects are mixed in the HDR image. An object of the present invention is to obtain an image display device capable of displaying each observation object in an easy-to-understand manner.
Further, according to the present invention, even if the difference in luminance between the plurality of observation objects mixed in the HDR image is slight, the difference in the shape and state of the plurality of observation objects can be easily visually confirmed. An object is to obtain an image display device.
Furthermore, an object of the present invention is to obtain an image display device that can optimize the luminance conversion of a region that needs to be visualized and can easily display the region that needs to be visualized.

  The image display device according to the present invention classifies pixels having luminance similar to that of adjacent pixels among pixels constituting an image having a luminance gradation exceeding the visible range into the same group. The brightness conversion target area setting means for setting the area where the pixels belonging to the group exist as the brightness conversion target area, and the brightness conversion target area for each brightness conversion target area set by the brightness conversion target area setting means And a luminance conversion means for converting the luminance of the pixels existing in the luminance conversion target area to the luminance in the visible range after removing the noise existing in the luminance conversion target area, and the area synthesis means by the luminance conversion means A synthesized image is generated by synthesizing images in one or more luminance conversion target areas in which the luminance of pixels is converted.

  According to the present invention, among pixels constituting an image having a luminance gradation exceeding the visible range, pixels whose luminance approximates that of adjacent pixels are classified into the same group and belong to the same group. The brightness conversion target area setting means for setting the area where the pixel exists as the brightness conversion target area and the brightness conversion target area set by the brightness conversion target area setting means are present in the brightness conversion target area. Luminance conversion means for converting the luminance of the pixels existing in the luminance conversion target area to luminance in the visible range after the noise is removed, and the area synthesis means uses the luminance conversion means to Since the image in one or more luminance conversion target areas converted from is synthesized and a composite image is generated, each observation is performed even when various observation objects are mixed in the image. There is an effect that can be displayed facilitate understanding of the elephant products.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the image display apparatus by Embodiment 1 of this invention. It is a block diagram which shows the inside of the brightness | luminance conversion object area | region setting part 3 of the image display apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the brightness | luminance conversion object area | region set by the brightness | luminance conversion object area | region setting part 3. FIG. It is explanatory drawing which shows the correspondence of the threshold value of clustering, and the number of pixels contained in a cluster. It is explanatory drawing which shows the example of a setting of a brightness | luminance conversion object area | region. It is explanatory drawing which shows an example of the histogram of the pixel value in a brightness | luminance conversion object area | region. It is explanatory drawing which shows an example of the brightness | luminance conversion result of the brightness | luminance conversion part. It is explanatory drawing which shows the example of a display of the synthesized image produced | generated by the area | region synthetic | combination part. It is a block diagram which shows the inside of the brightness | luminance conversion object area | region setting part 3 of the image display apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows the brightness | luminance of the pixel in the attention range of FIG. It is explanatory drawing which shows the example of a setting of a brightness | luminance conversion object area | region. It is a block diagram which shows the inside of the brightness | luminance conversion object area | region setting part 3 of the image display apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows the example of histogram evaluation by the clustering part 3i. It is explanatory drawing which shows the luminance conversion process by the luminance conversion part. It is explanatory drawing which shows the example of arrangement | positioning of an observation target object. It is explanatory drawing which shows the luminance distribution in a HDR image. It is explanatory drawing which shows the luminance distribution after implementing luminance conversion using a TMO system. It is explanatory drawing which shows a linear transformation result image.

Embodiment 1 FIG.
1 is a block diagram showing an image display apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the HDR image storage unit 1 is, for example, a synthetic aperture radar as a high dynamic range image (HDR image) that is an image having a luminance gradation that exceeds the human visible range and exceeds the number of visible gradations. It is a recording medium such as a memory that stores a SAR image that is a radio wave image acquired by using a high dynamic range camera.
Note that when the number of vertical pixels is M and the number of horizontal pixels is N, the HDR image holds M × N pieces of luminance information, and the luminance information of one pixel can be expressed by, for example, 8 bytes. Expressed as an integer. The luminance information indicates that the larger the integer value, the higher the luminance.

The conversion area range designating unit 2 accepts designation of the range of the area to be subjected to luminance conversion in the HDR image stored by the HDR image storage unit 1, and converts the HDR image in the conversion area to the luminance conversion target area setting unit 3. Execute the process to output to. The conversion area range designation unit 2 constitutes a conversion area range designation unit.
Although FIG. 1 does not show a specific configuration example of the conversion area range designation unit 2, the conversion region range designation unit 2 is configured by, for example, hardware as described below.
A memory interface that reads an HDR image from the HDR image storage unit 1 A man-machine interface that accepts specification of a range of areas for luminance conversion (for example, a keyboard and a mouse)
An image processing unit that cuts out an image in the conversion region from the HDR image and outputs the image to the luminance conversion target region setting unit 3 (for example, a semiconductor integrated circuit in which a CPU is mounted)
Here, an example is shown in which the conversion area range designation unit 2 is configured from a memory interface, a man-machine interface, and an image processing unit, but this is only an example. For example, the conversion area range designation unit 2 is a one-chip microcomputer. Etc. may be comprised.

The luminance conversion target area setting unit 3 sets adjacent pixels and pixels whose luminance is approximated to the same group among the pixels constituting the HDR image in the conversion area output from the conversion area range specifying unit 2. A process of classifying and setting a region where pixels belonging to the same group exist as a luminance conversion target region is performed. The luminance conversion target area setting unit 3 constitutes luminance conversion target area setting means.
In FIG. 1, the luminance conversion target area setting unit 3 sets N luminance conversion target areas (1) to (N), and sets the N luminance conversion target areas (1) to (N) as the luminance conversion unit 4. Shows an output example.
A specific configuration example of the luminance conversion target area setting unit 3 will be described later.

  The luminance conversion unit 4 removes noise existing in the luminance conversion target region (n) for each of the luminance conversion target regions (1) to (N) set by the luminance conversion target region setting unit 3, A process of converting the luminance of the pixels existing in the luminance conversion target region (n) into luminance in the visible range (for example, 255 gradations) is performed by linear luminance compression. However, n = 1, 2,..., N.

Alternatively, the luminance conversion unit 4 removes noise existing in the luminance conversion target region (n) for each of the luminance conversion target regions (1) to (N) set by the luminance conversion target region setting unit 3. From the average value μ and the variance value σ of the pixels existing in the luminance conversion target area (n), the luminance conversion is performed in the visible range from the average value μ and the variance value σ. A range S (n) (for example, a range that is three times the variance value σ above and below the average value μ centered on the average value μ) is determined, and brightness conversion is performed using the average value μ and the variance value σ. A process of converting the luminance of the pixels existing in the target area (n) into luminance within the luminance conversion range S (n) is performed.
In addition, the brightness | luminance conversion part 4 comprises the brightness | luminance conversion means.

The memory 4 a-1 is a recording medium that temporarily stores an image of the brightness conversion target area (1) set by the brightness conversion target area setting unit 3.
The memory 4a-2 is a recording medium that temporarily stores an image of the luminance conversion target area (2) set by the luminance conversion target area setting unit 3.
The memories 4a-N are recording media that temporarily store an image of the luminance conversion target area (N) set by the luminance conversion target area setting unit 3.

  The luminance conversion processing unit 4b-1 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted, and after removing noise existing in the luminance conversion target region (1), the luminance conversion target region (1 ) Is performed to convert the luminance of the pixels existing in the parenthesis into luminance within the luminance conversion range S (1) by linear luminance compression. Alternatively, after removing the noise existing in the luminance conversion target area (1), the average value μ and the variance value σ of the luminance of the pixels existing in the luminance conversion target area (1) are calculated. The luminance conversion range S (1) is determined from the average value μ and the variance value σ, and the luminance of the pixels existing in the luminance conversion target region (1) is determined using the average value μ and the variance value σ. A process of converting to the luminance within the luminance conversion range S (1) is performed.

  The luminance conversion processing unit 4b-2 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted. After removing noise existing in the luminance conversion target region (2), the luminance conversion target region (2 ) Is performed to convert the luminance of the pixels existing in () into luminance within the luminance conversion range S (2) by linear luminance compression. Alternatively, after removing noise existing in the luminance conversion target area (2), the average value μ and the variance value σ of the luminance of the pixels existing in the luminance conversion target area (2) are calculated. The luminance conversion range S (2) is determined from the average value μ and the variance value σ, and the luminance of the pixels existing in the luminance conversion target region (2) is determined using the average value μ and the variance value σ. A process of converting to luminance within the luminance conversion range S (2) is performed.

  The luminance conversion processing unit 4b-N is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted. After removing noise existing in the luminance conversion target region (N), the luminance conversion target region (N ) Is performed to convert the luminance of the pixels existing in () into luminance within the luminance conversion range S (N) by linear luminance compression. Alternatively, after removing noise existing in the luminance conversion target region (N), the average value μ and the variance value σ of the luminance of the pixels existing in the luminance conversion target region (N) are calculated. The luminance conversion range S (N) is determined from the average value μ and the variance value σ, and the luminance of the pixels existing in the luminance conversion target region (N) is determined using the average value μ and the variance value σ. A process of converting to luminance within the luminance conversion range S (N) is performed.

The memory 4c-1 is a recording medium that temporarily stores an image after luminance conversion of the luminance conversion target area (1).
The memory 4c-2 is a recording medium that temporarily stores an image after luminance conversion of the luminance conversion target area (2).
The memory 4c-N is a recording medium that temporarily stores an image after luminance conversion in the luminance conversion target area (N).
FIG. 1 shows an example in which the luminance conversion unit 4 includes the memories 4a-1 to 4a-N, the luminance conversion processing units 4b-1 to 4b-N, and the memories 4c-1 to 4c-N. Is merely an example. For example, the luminance conversion unit 4 may be configured by a one-chip microcomputer.

The area synthesizing unit 5 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and the luminance conversion target areas (1) to (N) stored in the memories 4c-1 to 4c-N. The image after the luminance conversion of () is synthesized to generate a synthesized image.
That is, the region composition unit 5 includes the luminance conversion processing unit 4b− out of the luminances of the pixels constituting the HDR image (the pixels constituting the HDR image in the conversion region output from the conversion region range specifying unit 2). The luminance of the pixels existing in the luminance conversion target areas (1) to (N) before the luminance is converted by 1-4b-N are converted by the luminance conversion processing units 4b-1 to 4b-N. A process of replacing with luminance is performed. The region composition unit 5 constitutes region composition means.

The composite image storage unit 6 is a recording medium such as a memory that temporarily stores the composite image generated by the region composition unit 5.
The stereoscopic display unit 7 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and illustrates the luminance of the pixels constituting the composite image stored in the composite image storage unit 6. The processing to display in three dimensions on the display. The stereoscopic display unit 7 constitutes stereoscopic display means.

  In FIG. 1, the HDR image storage unit 1, the conversion region range specification unit 2, the luminance conversion target region setting unit 3, the luminance conversion unit 4, the region composition unit 5, the composite image storage unit 6, and the three-dimensional structure that are components of the image display device. Although the example in which the display unit 7 is configured by dedicated hardware has been shown, when the image display device is configured by a computer, the conversion area range specifying unit 2, the luminance conversion target area setting unit 3, the luminance A program indicating the processing contents of the conversion unit 4, the region composition unit 5, and the stereoscopic display unit 7 may be stored in a memory of a computer, and the CPU of the computer may execute the program stored in the memory.

FIG. 2 is a configuration diagram showing the inside of the luminance conversion target area setting unit 3.
In FIG. 2, the smoothing processing unit 3 a performs a process of smoothing the luminance of the pixels constituting the HDR image in the conversion area output from the conversion area range specifying unit 2.
The tangential surface processing unit 3b performs tangential surface processing on the pixels whose luminance has been smoothed by the smoothing processing unit 3a.
The differential operator processing unit 3c calculates the gradient of the pixel value in the HDR image after the ridge surface processing by performing the differential operator processing on the pixel on which the ridge surface processing is performed by the ridge surface processing unit 3b.

The group classification processing unit 3d selects one pixel group in which pixels whose gradient of the pixel value calculated by the differential operator processing unit 3c is equal to or less than a threshold among the pixels constituting the HDR image after the ridge face processing are collected. Implement the process of classifying into one group.
However, the group classification processing unit 3d performs the process of optimizing the threshold before performing the process of classifying into one group.
That is, the group classification processing unit 3d temporarily sets a threshold value to be compared with the gradient of the pixel value calculated by the differential operator processing unit 3c, and the number of pixels belonging to one group increases rapidly while sequentially decreasing the threshold value. A process of searching for the immediately preceding threshold (optimum threshold) is performed.

The luminance conversion target area setting processing unit 3e performs processing for setting a circumscribed area (for example, circumscribed rectangle, circumscribed rectangle, circumscribed circle) including pixels belonging to the group classified by the group classification processing unit 3d as the luminance conversion target area. To do.
In FIG. 2, the luminance conversion target area setting unit 3 includes a smoothing processing unit 3a, a ridge surface processing unit 3b, a differential operator processing unit 3c, a group classification processing unit 3d, and a luminance conversion target region setting processing unit 3e. Although an example is shown, this is only an example, and for example, the luminance conversion target area setting unit 3 may be configured by a one-chip microcomputer.

Next, the operation will be described.
When the conversion area range specifying unit 2 reads an HDR image from the HDR image storage unit 1 as an image having a luminance gradation exceeding the human visible range (for example, 255 gradations), the conversion area range specifying unit 2 displays the HDR image on a display (not shown). indicate.
At this time, it is assumed that the maximum luminance of the pixels constituting the HDR image is L _MAX and the minimum luminance is L _MIN .
Also, let L (i, j) be the luminance of a pixel existing at a position (i, j) advanced i pixels downward from the origin and j pixels rightward with the upper left of the HDR image as the origin.
In this case, assuming that the luminance of the pixel displayed on the display by the conversion area range specifying unit 2 is, for example, 255 gradations, the luminance of the pixel displayed on the display is represented by the following L ′ (i, j) (however, the decimal part is rounded down) Integer value).
L ′ (i, j) = L (i, j) × (L _MAX −L _MIN ) / 255

When the HDR image is displayed on the display, the conversion area range designation unit 2 performs a process of accepting designation of the area range for luminance conversion.
That is, when the user confirms the HDR image displayed on the display, the range of the area for luminance conversion is designated by operating the keyboard or mouse in the conversion area range designation unit 2.
Thereby, the conversion area range designation | designated part 2 receives a user's designation | designated, and sets the range of the area | region which performs luminance conversion.
FIG. 1 shows an example in which an area surrounded by a dotted line is designated as a range of an area for performing luminance conversion.
Note that the shape of the range to be specified may be a rectangular shape or a circular shape. Alternatively, the area may be specified by a broken line.

When the conversion area range specifying unit 2 sets the range of the area for luminance conversion, the conversion area range specifying unit 2 outputs the HDR image in the conversion area to the luminance conversion target area setting unit 3.
At this time, the luminance information of the HDR image output to the luminance conversion target region setting unit 3 is not an integer of 255 gradations displayed on the display, but in the original HDR image stored by the HDR image storage unit 1. Information on luminance L (i, j).
The luminance information also includes information on the position (i, j) of the pixels constituting the original HDR image.

When the luminance conversion target region setting unit 3 receives the HDR image in the conversion region (for example, the image of the region surrounded by the dotted line in FIG. 1) from the conversion region range specifying unit 2, the luminance conversion target region setting unit 3 configures the HDR image in the conversion region. Among pixels that are adjacent to each other, pixels that are similar in luminance to those of adjacent pixels are classified into the same group, and a region where pixels belonging to the same group exist is set as a luminance conversion target region.
That is, the luminance conversion target area setting unit 3, as shown in FIG. 3, pixels having the same luminance from the HDR image in the conversion area (for example, the image of the area surrounded by the dotted line in FIG. 1). The neighboring area (area surrounded by a dotted line) including “” is set as a separate luminance conversion target area.

Hereinafter, the processing content of the brightness conversion target area setting unit 3 will be described in detail.
First, when the smoothing processing unit 3a of the luminance conversion target region setting unit 3 receives the HDR image in the conversion region from the conversion region range specifying unit 2, the smoothing processing unit 3a smoothes the luminance of the pixels constituting the HDR image in the conversion region. Turn into.
That is, the smoothing processing unit 3a performs a filter process called moving average in order to reduce noise (a portion with high luminance appearing in a small area) included in the HDR image.
Specifically, the luminance values of 9 pixels of 3 × 3 pixels in the vertical and horizontal directions centering on the pixel to be converted are respectively multiplied by “1/9”, and the sum of the luminance values after the multiplication is converted to the pixel. This is the value of the later luminance.
As a result, as shown in FIG. 2, the high luminance (white dots) of several fine pixels disappears, so that noise is reduced.

In this example, the smoothing processing unit 3a performs a filtering process called moving average. However, among the pixels constituting the HDR image in the transform area, a preset threshold (reference luminance) is shown. It is also possible to search for a pixel having higher luminance than (level) and convert the luminance of the pixel to the threshold value.
If each pixel is redisplayed, the maximum luminance of the converted pixel is L _MAX , the minimum luminance is L _MIN , and the display luminance of each pixel is L (i, j) × (L _MAX −L _MIN ) / 255. As shown in the smoothed display example of FIG. 2, an image with reduced noise can be displayed.

Next, the climax surface processing unit 3b of the luminance conversion target area setting unit 3 performs tangential surface processing on a pixel (note: not 255 gradation) whose luminance has been smoothed by the smoothing processing unit 3a. carry out.
An HDR image, which is a radio wave image acquired using a synthetic aperture radar, may appear as a striped pattern with respect to a uniform state portion on the ground.
The tangential surface process is a process of filling a black portion of a stripe (a groove such as a dent or stripe when displaying luminance), and in the tangential surface process, it is located around the pixel to be converted. Among the 8 pixel luminances, the maximum luminance is converted into the luminance of the pixel to be converted.
If each pixel is redisplayed, the maximum luminance of the converted pixel is L _MAX , the minimum luminance is L _MIN , and the display luminance of each pixel is L (i, j) × (L _MAX −L _MIN ) / 255. As shown in the tangential surface processing display example of FIG. 2, it is possible to display each observation object as if it was painted white.

Next, the differential operator processing unit 3c of the luminance conversion target area setting unit 3 performs differential operator processing on the pixels that have been subjected to the ridge surface processing by the ridge surface processing unit 3b.
That is, the differential operator processing unit 3c performs filter processing using an 8-direction Laplacian filter in order to calculate the gradient of the pixel value in the HDR image after the ridge surface processing.
By using the gradient of the pixel value, it is possible to detect a region where the pixel value is changed compared to the surroundings, regardless of the magnitude of the pixel value in the HDR image. For this reason, it is possible to simultaneously detect a high luminance region and a low luminance region which are mixed.
If each pixel is redisplayed, the maximum luminance of the converted pixel is L _MAX , the minimum luminance is L _MIN , and the display luminance of each pixel is L (i, j) × (L _MAX −L _MIN ) / 255. As shown in the differential operator display example of FIG. 2, it becomes possible to raise the edge portion where the brightness changes.

Next, the group classification processing unit 3d of the luminance conversion target region setting unit 3 uses the gradient of the pixel value calculated by the differential operator processing unit 3c among the pixels constituting the HDR image after the ridge face processing as a threshold value. Processing for classifying the pixel group in which the following pixels are gathered into one group is performed.
Hereinafter, the processing content of the group classification | category process part 3d is demonstrated concretely.

First, the group classification processing unit 3d sets the absolute value of the gradient calculated by the differential operator processing unit 3c as the pixel value in order to set the starting point of the region expansion method for estimating the region where the observation target exists. By clustering the change areas in the image, an area in which the change in the pixel value is large (area where the observation target exists) is detected in the HDR image.
However, even in the region where the observation object exists, points with large gradients may not be continuous, and clustering of discrete points is necessary. Ask for.
The group classification processing unit 3d sets the starting point of the region expansion method to the point closest to the center of gravity of the points included in the cluster (same group).

Since the pixel value gradient clustering result includes both high and low pixel values, the region where the observation target is present is estimated by the region expansion method.
In the HDR image, when the pixel value is expressed in the height direction by the ridge surface processing, the region where the observation target object exists has a convex hull shape. If the value is equal to or greater than the set threshold value, the region where the observation target object exists is estimated by the process of including the pixel in the cluster.
However, in the estimation of the region where the observation object exists, the number of pixels included in the cluster varies depending on the clustering threshold.
The threshold value for clustering refers to the boundary value between a pixel other than the pixel indicating the observation object and the pixel indicating the observation object. Depending on the state of the ground surface, the pixel value of the area where there is no observation object indicates a constant value. As a result, the threshold cannot be determined.

Therefore, the group classification processing unit 3d optimizes the clustering threshold as follows.
FIG. 4 is an explanatory diagram showing the correspondence between the threshold value for clustering and the number of pixels included in the cluster.
In FIG. 4, the vertical axis indicates the number of pixels included in the cluster, the horizontal axis indicates the threshold value for clustering, and the threshold value decreases from left to right in the figure.

Since it is estimated that a pixel having a higher luminance than the surrounding pixels is a pixel indicating the observation target, as shown in FIG. 4, when the threshold value is decreased, the number of pixels included in the cluster increases.
Further, when the threshold value is decreased, pixels other than the pixels indicating the observation target are also regarded as pixels indicating the observation target, and the number of pixels included in the cluster increases rapidly.
The group classification processing unit 3d sets the threshold immediately before the number of pixels included in the cluster rapidly increases as the clustering threshold in the region expansion method.
In the example of FIG. 4, when the threshold value A is used, the region where the observation target object exists cannot be clustered. When the threshold value C is used, the range other than the observation target object is clustered and included in the cluster. Since the number of pixels to be increased has increased rapidly, the threshold value B immediately before the C threshold value is set as the clustering threshold value.
Here, a method for automatically determining the threshold B as a threshold to be clustered will be described.
The threshold value is a variable (for example, x), and the number of pixels of the cluster obtained as a result of clustering using the threshold value x is u (x). Based on the difference method, the difference quotient of x1 and x2 is (u (x2) -u (x1)) / (x2-x1).
The difference between the values of x1 and x2 is Δx, and the second-order difference quotient is {u (x + 2Δx) -2u (x + Δx) + u (x)} / Δx ^ 2.
The value of Δx is determined in advance as a constant value, the value of x is changed, and the second-order difference quotient corresponding to x is calculated and recorded. In the case of the threshold value and the number of pixels shown in FIG. 4, the second-order difference quotient is the minimum (minus value) when x = B. The reason for taking a negative value is that the increase in the number of pixels is maximized as the threshold value decreases. In this way, the inflection point B is obtained.

  When the threshold value for clustering is optimized, the group classification processing unit 3d optimizes the gradient of the pixel value calculated by the differential operator processing unit 3c among the pixels constituting the HDR image after the ridge face processing. A pixel group in which pixels that are equal to or less than a certain threshold are gathered is classified into one group.

Next, the luminance conversion target region setting processing unit 3e of the luminance conversion target region setting unit 3 performs processing for setting a circumscribed region including pixels belonging to the group classified by the group classification processing unit 3d as the luminance conversion target region. To do.
FIG. 5 is an explanatory diagram showing an example of setting the luminance conversion target area.
In the example of FIG. 5, the dotted line portion is set as the luminance conversion target area.
That is, a dotted line portion (a circumscribed rectangle) including black luminance pixels belonging to one group is set as a luminance conversion target region.
In this example, the circumscribed area is a circumscribed rectangle, but the circumscribed area may be a circumscribed rectangle or a circumscribed circle. Further, the pixel range may be a certain distance from the black luminance pixels belonging to the group.

In FIG. 5, the number added to the dotted line is the number of the group assigned for the purpose of explanation. The group of group number 8 becomes a donut shape in which the group of group number 2 is hollowed out.
In FIG. 5, the groups are classified into 10 groups. However, the joining process between the groups may be performed.
In the group combining process, a group in which the distance between pixels belonging to a group is equal to or smaller than a certain value and the average value and the variance value of the pixels belonging to the group read from the HDR image fall within a certain difference is the same group. It is processing to.
In the example of FIG. 5, it is assumed that the group of group number 3, group number 4, group number 5, group number 6, and group number 7 is the same group.

When the luminance conversion target region setting processing unit 3e sets N luminance conversion target regions (1) to (N) as described above, the luminance conversion target region setting processing unit 3e exists in the luminance conversion target regions (1) to (N). The luminance L (i, j) of the pixel being stored is stored in the memories 4a-1 to 4a-N of the luminance conversion unit 4.
Needless to say, the luminance L (i, j) of the pixel stored in the memories 4a-1 to 4a-N is not only information indicating the luminance of the pixel, but also the original HDR in which the pixel exists. Information indicating the position (i, j) in the image is also included.

When the luminance conversion unit 4 receives the luminance L (i, j) of the pixels existing in the luminance conversion target regions (1) to (N) from the luminance conversion target region setting unit 3, the luminance conversion target region setting unit For each of the luminance conversion target areas (1) to (N) set in step 3, noise existing in the luminance conversion target area (n) is removed.
That is, the luminance conversion unit 4 performs noise reduction on a small region in which pixels whose luminance is higher than a predetermined threshold as compared to the luminance of surrounding pixels among the pixels existing in the luminance conversion target region (n). Is determined to be an overlapping region.

Here, FIG. 6 is an explanatory diagram illustrating an example of a histogram of pixel values in the luminance conversion target region.
The luminance conversion unit 4 obtains an inflection point at which the frequency gradient becomes flat before the frequency decreases as the pixel value increases, and estimates a pixel whose luminance is higher than the pixel value at the inflection point as noise. Yes.
In the example of FIG. 6, the threshold for estimating a pixel as noise is “1935”.
Hereinafter, a method for automatically obtaining a threshold for estimating the noise will be described.
In FIG. 6, the horizontal axis indicates the range of pixel values.
When the value of one scale on the horizontal axis in FIG. 6 is 5, the number of pixels whose pixel values are within 5 from the scale value is indicated on the vertical axis, and is indicated by a vertical bar (referred to as a bin) on the graph.
Similar to the method of obtaining the inflection point in FIG. 4, the inflection point is also obtained in FIG.
Let the value of the pixel value of one bin be a variable (for example, x). In the figure, when the horizontal axis memory width is 5, for example, the number of pixels of pixel values x to x + 5 is g (x).
Based on the difference method, the difference quotient of x1 and x2 is (g (x2) −g (x1)) / (x2−x1).
If the difference between the values of x1 and x2 is Δx, the difference quotient is {g (x + Δx) −g (x)} / Δx.
The value of Δx is determined in advance as a constant value, the value of x is changed, and the second-order difference quotient corresponding to x is calculated and recorded.
Assuming that the value of Δx and the amount of difference that changes the value of x are integer multiples of the bin width, a threshold for estimating noise can be obtained using only the values shown in the graph of FIG. If the value of Δx or the amount of difference that changes the value of x is not an integral multiple of the bin width, the histogram needs to be recalculated.
A pixel having a high pixel value is regarded as noise with a pixel value x1 where the difference quotient value satisfies all of the following conditions as a boundary.
Condition 1 The value of g (x) is less than or equal to a threshold value separately defined for x where x1 <x Condition 2 The number of pixels indicating the pixel value satisfying x <x1 is the total number of pixels x separately determined ratio (threshold value) more than.
Condition 3 For a predetermined difference amount Δx2,
In the interval where the difference quotient value at x where x−Δx2 <x <x1 is smaller than the difference quotient value at x where x1 <x <x + Δx2, the difference quotient is greater than or equal to a predetermined threshold. value

When the luminance conversion unit 4 estimates a region where noise is superimposed, the luminance conversion unit 4 uses a low-pass filter to remove the noise so that a change in pixel value that is less than a threshold to be estimated as noise can be visually recognized.
The luminance conversion unit 4 performs processing for converting the luminance of the pixels existing in the luminance conversion target region (n) into luminance in the visible range (for example, 255 gradations) by linear luminance compression after removing the noise. carry out.
In this example, the luminance conversion unit 4 converts the luminance of the pixel by linear luminance compression. However, the luminance of the pixel may be converted as follows.

The luminance conversion processing unit 4b-1 of the luminance conversion unit 4 reads the image of the luminance conversion target region (1) set by the luminance conversion target region setting unit 3 from the memory 4a-1, and stores the image in the luminance conversion target region (1). The average value μ and the variance value σ of the luminance of the pixels existing in the are calculated.
Next, the luminance conversion processing unit 4b-1 determines the luminance conversion range S (1) from the average value μ and the variance value σ.
For example, a range that is three times the variance value σ above and below the average value μ with the average value μ as the center is determined as the luminance conversion range S (1).
Lower limit value of S (1) = μ−3 × σ
Upper limit value of S (1) = μ + 3 × σ

Next, the luminance conversion processing unit 4b-1 expresses the luminance conversion range S (1) with 255 gradations, and the luminance L (i, j) of the pixels existing in the luminance conversion target region (1). Is converted to the luminance cL (i, j) within the luminance conversion range S (1).
cL (i, j) = (L (i, j) − (μ−3 × σ)) / (6 × σ / 255)
However, the luminance cL (i, j) is a value obtained by removing the decimal point from the value on the right side of the above equation.
When the luminance of the pixel existing in the luminance conversion target area (1) is lower than the lower limit value of the luminance conversion range S (1), the luminance of the pixel is converted to “0” and the luminance conversion is performed. If it is higher than the upper limit value of the range S (1), the luminance of the pixel is converted to “255”.

The luminance conversion processing unit 4b-2 reads the image of the luminance conversion target region (2) set by the luminance conversion target region setting unit 3 from the memory 4a-2, and exists in the luminance conversion target region (2). An average value μ and a variance value σ of the luminance of the pixels are calculated.
Next, similarly to the luminance conversion processing unit 4b-1, the luminance conversion processing unit 4b-2 determines the luminance conversion range S (2) from the average value μ and the variance value σ, and the luminance conversion range S ( 2) is expressed by 255 gradations, and the luminance L (i, j) of the pixel existing in the luminance conversion target region (2) is changed to the luminance cL (i, j) in the luminance conversion range S (2). Convert.

The luminance conversion processing unit 4b-N reads an image of the luminance conversion target region (N) set by the luminance conversion target region setting unit 3 from the memory 4a-N, and exists in the luminance conversion target region (N). An average value μ and a variance value σ of the luminance of the pixels are calculated.
Next, similarly to the luminance conversion processing unit 4b-1, the luminance conversion processing unit 4b-N determines the luminance conversion range S (N) from the average value μ and the variance value σ, and the luminance conversion range S ( N) is expressed by 255 gradations, and the luminance L (i, j) of the pixel existing in the luminance conversion target region (N) is changed to the luminance cL (i, j) in the luminance conversion range S (N). Convert.

As described above, the luminance conversion processing units 4b-1 to 4b-N determine the luminance conversion ranges S (1) to S (N) by the same method, and are within the luminance conversion target regions (1) to (N). Is converted into luminance cL (i, j) within the luminance conversion range S (1) to S (N), but the luminance conversion target region (1) Since the average value μ and the variance value σ of the luminance in (N) are different for each luminance conversion target region, different luminance conversion ranges S (1) to S (N) are determined for each luminance conversion target region. The ranges S (1) to S (N) are conversion ranges suitable for the luminance conversion target region.
Therefore, optimal brightness conversion can be realized for each of the brightness conversion target areas (1) to (N).

FIG. 7 is an explanatory diagram illustrating an example of the luminance conversion result of the luminance conversion unit 4.
As shown in FIG. 7, it can be seen that the luminance conversion results represent the detailed details of the original observation data.
The luminance cL (i, j) of the image after luminance conversion of the luminance conversion target region (1) is stored in the memory 4c-1, and the luminance cL (i, j of the image after luminance conversion of the luminance conversion target region (2) is stored. j) is stored in the memory 4c-2.
Further, the luminance cL (i, j) of the image after the luminance conversion of the luminance conversion target region (N) is stored in the memory 4c-N.

The area synthesizing unit 5 synthesizes the luminance converted images of the luminance conversion target areas (1) to (N) stored in the memories 4c-1 to 4c-N to generate a synthesized image.
That is, when the region composition unit 5 receives an HDR image (for example, an image of a region surrounded by a dotted line in FIG. 1) in the conversion region from the conversion region range specifying unit 2, the pixels constituting the HDR image Each pixel is displayed on a display (not shown) where L _{MAX is} the maximum luminance, L _{MIN is} the minimum luminance, and L (i, j) × (L _MAX −L _MIN ) / 255 is the display luminance of each pixel.
Next, the region composition unit 5 reads the luminance-converted images of the luminance conversion target regions (1) to (N) stored in the memories 4c-1 to 4c-N of the luminance conversion unit 4 and displays them on the display. A process of replacing the luminance L (i, j) of the pixels in the luminance conversion target areas (1) to (N) with the luminance cL (i, j) of the image after the luminance conversion is performed.
The image after the luminance conversion by the region synthesis unit 5 is stored in the synthesized image storage unit 6 as a synthesized image.
FIG. 8 is an explanatory diagram showing a display example of a composite image generated by the region composition unit 5, and this composite image is displayed on a display (not shown).

  Here, the region composition unit 5 includes images other than the luminance conversion target regions (1) to (N) in the composite image among the HDR images in the transform region set by the transform region range designating unit 2. However, an image other than the luminance conversion target areas (1) to (N) is not included in the composite image, and only the images of the luminance conversion target areas (1) to (N) are combined to generate a composite image. It may be.

When the region composition unit 5 stores the composite image in the composite image storage unit 6, the stereoscopic display unit 7 reads the composite image from the composite image storage unit 6 and, as shown in FIG. 1, the pixels constituting the composite image Is displayed three-dimensionally on a display (not shown).
That is, the stereoscopic display unit 7 has the luminance at the pixel position (i, j) of the synthesized image (the luminance of the pixels constituting the luminance conversion target areas (1) to (N) is cL (i, j), The luminance of the pixels constituting the image other than the luminance conversion target areas (1) to (N) is L (i, j), but here, for convenience of explanation, all the pixels constituting the composite image Is represented by cL (i, j)), and a point having three-dimensional coordinates (a × i, a × j, cL (i, j)) is generated.
However, a is an arbitrary real number that can be set by the user.

When the three-dimensional display unit 7 generates a point having three-dimensional coordinates (a × i, a × j, cL (i, j)), the point is displayed on a display (not shown) as shown in FIG.
FIG. 1 shows an example in which a portion having a high central luminance is displayed. However, the three-dimensional viewpoint position can be freely changed.
Note that the three-dimensional synthesized image by the stereoscopic display unit 7 and the two-dimensional synthesized image by the region synthesizing unit 5 may be superimposed and displayed side by side.

  As apparent from the above, according to the first embodiment, among the pixels constituting the HDR image, the pixels whose luminance is similar to that of the adjacent pixels are classified into the same group, and the same group. A luminance conversion target area setting unit 3 that sets areas where pixels belonging to the luminance conversion target areas (1) to (N) exist, and a luminance conversion target area (1) set by the luminance conversion target area setting unit 3 ) To (N), the noise existing in the luminance conversion target area (n) is removed, and the luminance of the pixel existing in the luminance conversion target area (n) is set within the visible range. A luminance conversion unit 4 that converts the luminance into a luminance cL (i, j), and the region synthesis unit 5 converts the images in the luminance conversion target regions (1) to (N), in which the luminance of the pixels is converted by the luminance conversion unit 4 Since it is composed to generate a composite image, It is is an effect which can be displayed or when, even if a different observation target object is mixed in the HDR image, clarity of each observation object contained in the HDR image.

  Further, according to the first embodiment, the conversion area range designation unit 2 designates a range of an area to be subjected to luminance conversion in the HDR image, and outputs an image in the area to the luminance conversion target area setting unit 3. With this configuration, it is possible to perform luminance conversion only in the range of an area in the HDR image that needs to be visualized (for example, an area that the user desires to display). It is possible to display an observation object in an area that needs to be visualized in an easy-to-understand manner without being affected by noise.

  According to the first embodiment, the stereoscopic display unit 7 is configured to stereoscopically display the luminance of the pixels constituting the synthesized image generated by the region synthesizing unit 5, so that it is mixed in the HDR image. Even if the difference in brightness between the plurality of observation objects is small, there is an effect that the difference in shape and state of the plurality of observation objects can be easily visually recognized.

  Further, according to the first embodiment, the luminance conversion target region setting unit 3 smoothes the luminance by the smoothing processing unit 3a that smoothes the luminance of the pixels constituting the HDR image, and the smoothing processing unit 3a. The ridge surface processing unit 3b that performs the tangential surface processing on the converted pixels and the differential operator processing for the pixels that have been subjected to the ridge surface processing by the ridge surface processing unit 3b The differential operator processing unit 3c for calculating the gradient of the pixel value in the subsequent HDR image and the gradient of the pixel value calculated by the differential operator processing unit 3c among the pixels constituting the HDR image after the ridge peak processing. A group classification processing unit 3d that classifies a pixel group in which pixels that are less than or equal to the threshold value are grouped into one group, and a circumscribed area that includes pixels belonging to the group classified by the group classification processing unit 3d. Since it is composed of the luminance conversion target region setting processing unit 3e that is set as the replacement target region, it is possible to perform luminance conversion for each region having similar luminance characteristics, and the observation target in each region There is an effect that objects can be displayed in an easy-to-understand manner.

  According to the first embodiment, the threshold value to be compared with the gradient of the pixel value calculated by the differential operator processing unit 3c is temporarily set before the group classification processing unit 3d performs the processing of classifying into one group. Since the threshold value (optimal threshold value) immediately before the number of pixels belonging to one group rapidly increases is searched while the threshold value is sequentially decreased, the pixel group related to the observation object is accurately set to one group. There is an effect that can be classified into.

  According to the first embodiment, the luminance conversion unit 4 is included in the luminance conversion target region (n) for each of the luminance conversion target regions (1) to (N) set by the luminance conversion target region setting unit 3. After removing the existing noise, the luminance cL (i, j) within the visible range is obtained by linear luminance compression of the luminance L (i, j) of the pixel existing in the luminance conversion target region (n). Therefore, it is possible to visually recognize changes in pixel values that are less than the threshold value estimated as noise, and to realize optimal luminance conversion for each of the luminance conversion target areas (1) to (N). Can do. As a result, there is an effect that the observation objects in the luminance conversion target areas (1) to (N) can be displayed in an easily understandable manner.

  According to the first embodiment, the luminance conversion unit 4 is included in the luminance conversion target region (n) for each of the luminance conversion target regions (1) to (N) set by the luminance conversion target region setting unit 3. After removing the existing noise, the average value μ and the variance value σ of the luminance L (i, j) of the pixels existing in the luminance conversion target region (n) are calculated, and the average value is calculated. Luminance conversion ranges S (1) to S (N) are determined within the visible range from μ and the variance value σ, and the average value μ and the variance value σ are used to determine the luminance conversion range (n). Is converted to the luminance cL (i, j) in the luminance conversion range S (n), so that the pixel value that does not satisfy the threshold value for estimating noise The change in brightness can be visually recognized, and the optimum brightness conversion is realized for each of the brightness conversion target areas (1) to (N). It is possible. As a result, there is an effect that the observation objects in the luminance conversion target areas (1) to (N) can be displayed in an easily understandable manner.

Embodiment 2. FIG.
FIG. 9 is a block diagram showing the inside of the luminance conversion target area setting unit 3 of the image display apparatus according to Embodiment 2 of the present invention.
In FIG. 9, the high-luminance small region removal unit 3 f compares the luminance of the pixels constituting the HDR image in the conversion region output from the conversion region range specification unit 2 with a preset threshold value, thereby obtaining the threshold value. A small area (a few pixels) in which pixels with higher luminance are gathered is detected, and the luminance of the pixels in the small area is changed to other luminance (for example, a pixel other than the small area and its small size). To replace the brightness of the pixel closest to the area).

The clustering unit 3g sequentially scans the luminance of the pixels constituting the HDR image after the replacement processing by the high luminance small region removing unit 3f, and classifies the pixels whose luminance is similar to the adjacent pixels into the same group. Perform the process.
The luminance conversion target region setting processing unit 3h performs processing for setting a circumscribed region (for example, circumscribed rectangle, circumscribed rectangle, circumscribed circle) including pixels belonging to the group classified by the clustering unit 3g as the luminance conversion target region.

In the first embodiment, the luminance conversion target region setting unit 3 includes the smoothing processing unit 3a, the ridge surface processing unit 3b, the differential operator processing unit 3c, the group classification processing unit 3d, and the luminance conversion target region setting processing unit 3e. However, this is merely an example. For example, as illustrated in FIG. 9, the luminance conversion target region setting unit 3 includes a high luminance small region removing unit 3 f, a clustering unit 3 g, and a luminance conversion target region setting. You may be comprised from the process part 3h.
Hereinafter, the processing content of the brightness conversion target area setting unit 3 will be described in detail.

When the high luminance small region removing unit 3f of the luminance conversion target region setting unit 3 receives the HDR image in the conversion region from the conversion region range specifying unit 2, the luminance of the pixels constituting the HDR image is set in advance. Compare brightness thresholds.
When detecting one or more pixels whose luminance is higher than the luminance threshold, the high luminance small area removing unit 3f groups those pixels if the distance between the pixels is shorter than a preset distance threshold.
Here, the comparison is made between the luminance of the pixels constituting the HDR image and the luminance threshold value, but 8 pixels other than the comparison target pixel among the peripheral 3 × 3 pixels including the comparison target pixel at the center. The average luminance value of the pixel may be compared with the luminance of the center pixel.

Next, if the number of pixels belonging to one group is less than a preset number threshold, the high luminance small area removing unit 3f superimposes noise on this group (small area having several pixels). Is recognized as a region (hereinafter referred to as “noise group”).
When the noise group is recognized, the high-luminance small area removing unit 3f removes noise superimposed on the HDR image by replacing the luminance of the pixels belonging to the noise group with other luminance.
For example, the luminance of a pixel belonging to a noise group is replaced with the luminance of a pixel that does not belong to the noise group and is located closest to the noise group.

As a result, the maximum luminance of the converted pixel is L _MAX , the minimum luminance is L _MIN , and the display luminance of each pixel is L (i, j) × (L _MAX −L _MIN ) / 255. For example, an image without noise can be displayed as in the display example of the high luminance small area removal result in FIG.
However, since the luminance width of the pixel after conversion is also large, an overall dark image may appear at this stage, and a portion that is difficult to view may appear.

When the clustering unit 3g receives the HDR image after the replacement process from the high luminance small region removing unit 3f, the clustering unit 3g sequentially scans the luminance of the pixels constituting the HDR image, and the luminance is approximated to the adjacent pixels. A process of classifying the pixels into the same group is performed.
Hereinafter, the processing content of the clustering unit 3g will be specifically described.

First, as shown in FIG. 9, the clustering unit 3g performs a one-dimensional process of extracting the brightness level of the pixels constituting the HDR image at positions A, B, C, D, and E.
In the example of FIG. 9, when moving the scanning lines in the order of A to E and taking out the level of luminance, the scanning line direction in A to E is set to the vertical direction, but the scanning line direction is set to the vertical direction. It is not limited, and it may be a lateral direction or an oblique direction.
The direction of the scanning line may be arbitrarily designated by the user, or the direction of the scanning line may be automatically determined.
The scanning lines are moved in the order of A to E. However, as shown in FIG. 9, the scanning lines are moved not only in a part but also in the entire HDR image in the conversion area output from the conversion area range specifying unit 2. It may be.

Here, FIG. 10 is an explanatory diagram showing the luminance of the pixels in the attention range of FIG.
As shown in FIG. 10, in the scanning line A, the luminance of the pixels is uniformly low, and the luminance of the pixels in the attention range is also low.
In the B, C, and D scanning lines, the luminance of the pixels in the attention range is high, and in the E scanning line, the luminance of the pixels in the attention range is low.

The clustering unit 3g groups the high-luminance portions of the target range in the B, C, and D scanning lines.
That is, the clustering unit 3g extracts a range surrounded by the A scanning line and the E scanning line, which have low luminance, in the target range, and sets the range as one group.
The determination of high luminance or low luminance is made by setting a predetermined threshold value in advance. Alternatively, the average value and variance value of the luminance values of the pixels on the scanning line are obtained, and the luminance that is equal to or higher than a certain multiple of the variance value is determined as high luminance from the average value, and the luminance that is equal to or lower than the certain multiple of the variance value is determined as low luminance You may make it do.
Alternatively, the same group may be used when the luminance difference between pixels adjacent to each other in the scanning direction and the direction orthogonal to the scanning direction is equal to or less than a predetermined threshold. In this case, in the clustering unit 3g, it is determined that a pixel whose luminance is not close to that of pixels adjacent vertically and horizontally does not belong to any group. A pixel position corresponding to the outer periphery of a connected component of pixels not belonging to a group is set as a boundary point.

Alternatively, in a method in which the same group is used when the luminance difference between pixels adjacent to each other in the scanning direction and the direction orthogonal to the scanning direction is equal to or less than a predetermined threshold value (hereinafter referred to as “kido”), If the number of pixels belonging to is less than a predetermined threshold value (hereinafter, this threshold value is expressed as “gasonum” and is larger than the threshold value for determining a noise region), the classification into the group is performed. Shall stop. At this time, the values of kids and gasonum are changed within a predetermined range, and clustering processing is performed for each of the kidso and gasonum values. Thus, the number of pixels belonging to each group and the number of pixels of a group of pixels not belonging to the group are obtained.
Due to the change of kido, it is possible to group regions where the change in the reflection characteristics of radio waves is within a certain range.
When clustering is performed by fixing the gasonum value and increasing the kido value, and using the gasonum value and the kido value that reduce the amount of change in the number of groups and the number of pixels belonging to each group, it is excessive due to the strength of the reflection intensity. Grouping is not done.

The luminance conversion target area setting processing unit 3h sets a circumscribed area (for example, circumscribed rectangle, circumscribed rectangle, circumscribed circle) including pixels belonging to the group classified by the clustering unit 3g as the luminance conversion target area.
FIG. 11 is an explanatory diagram showing an example of setting the luminance conversion target area.
In the example of FIG. 11, the dotted line portion is set as the luminance conversion target area.
That is, a dotted line portion (a circumscribed rectangle) including pixels belonging to one group is set as a luminance conversion target region.
The group of group number 2 includes a pipe-like line in the outer frame, but the straight line for extracting the brightness does not become flat until it passes through the outer frame, and is grouped in an enclosed shape. ing.

When the luminance conversion target area setting processing unit 3h sets N luminance conversion target areas (1) to (N), the luminance L of the pixels existing in the luminance conversion target areas (1) to (N) is set. (I, j) is stored in the memories 4 a-1 to 4 a -N of the luminance conversion unit 4.
Needless to say, the luminance L (i, j) of the pixel stored in the memories 4a-1 to 4a-N is not only information indicating the luminance of the pixel, but also the original HDR in which the pixel exists. Information indicating the position (i, j) in the image is also included.
Further, as a process of the luminance conversion unit 4, a process of setting the pixel at the boundary point to a predetermined color is performed.

As is apparent from the above, even when the luminance conversion target region setting unit 3 includes the high luminance small region removing unit 3f, the clustering unit 3g, and the luminance conversion target region setting processing unit 3h, the same as in the first embodiment. In addition, it is possible to perform luminance conversion for each region having similar luminance characteristics, so that the observation target object in each region can be displayed in an easily understandable manner.
According to the second embodiment, the high-luminance small area removing unit 3f determines the noise area, replaces the pixel corresponding to the noise with the same color as the surrounding pixels, and the clustering unit 3g uses the adjacent pixels to determine the luminance. A part having a large change increase / decrease is set as a non-group, and a boundary point around it is detected, and the luminance conversion unit 4 sets the pixel at the boundary point to a specific color. Thereby, for example, when there is a small steel structure that reflects radio waves well in the grassland, the luminance change of the pixel is large only in this part, so that only the periphery of this part can be easily displayed in a specific color. There is an effect.
Further, since the clustering unit 3g can automatically determine the variables necessary for clustering, it is possible to perform luminance conversion regardless of the skill of the operator.

Embodiment 3 FIG.
FIG. 12 is a block diagram showing the inside of the luminance conversion target area setting unit 3 of the image display device according to Embodiment 3 of the present invention. In the figure, the same reference numerals as those in FIG. To do.
The clustering unit 3i draws a histogram around the designated point in the HDR image after the replacement processing by the high-intensity small region removing unit 3f, and is surrounded by a point where the gradient of the histogram becomes flat (a region including the designated point as the center). ) Is set to one group.

In the first embodiment, the luminance conversion target region setting unit 3 includes the smoothing processing unit 3a, the ridge surface processing unit 3b, the differential operator processing unit 3c, the group classification processing unit 3d, and the luminance conversion target region setting processing unit 3e. However, this is merely an example. For example, as shown in FIG. 12, the luminance conversion target region setting unit 3 has a high luminance small region removing unit 3f, a clustering unit 3i, and a luminance conversion target region setting. You may be comprised from the process part 3h.
Hereinafter, the processing content of the brightness conversion target area setting unit 3 will be described in detail.

  The high luminance small region removing unit 3f of the luminance conversion target region setting unit 3 recognizes the noise group and replaces the luminance of the pixels belonging to the noise group with other luminance, as in the second embodiment. , Noise superimposed on the HDR image is removed.

When the clustering unit 3i receives the HDR image after the replacement process from the high-luminance small region removing unit 3f, the clustering unit 3i draws a histogram around the designated point in the HDR image.
In the example of FIG. 12, point A and point B are designated as designated points.
The designated point may be arbitrarily designated by the user, may be determined at regular intervals, or the position may be determined by a random number.
As shown in FIG. 13, the histogram is drawn by taking the designated point in the HDR image as the center, taking the distance from the pixel of the designated point on the horizontal axis and the luminance on the vertical axis. The gradient of the histogram decreases as the distance from the designated point increases.
In the example of FIG. 13, the distance between the direction S and the direction T is shown. The direction S and the direction T are orthogonal directions.
In addition, in the example of FIG. 13, the distance in the two directions of the direction S and the direction T is shown, but a distance in three or more directions may be shown.

When the clustering unit 3i draws the histogram, the clustering unit 3i specifies the point in the direction S and the point in the direction T in which the gradient of the histogram becomes flat, and the section surrounded by the point in the direction S and the point in the direction T (centering on the specified point). Are set as one group.
However, when a plurality of groups including the same point are set, the group that is included is deleted while leaving the group that includes it as much as possible.

When the clustering unit 3i sets a group, the luminance conversion target region setting processing unit 3h sets a circumscribed region including pixels belonging to the group (for example, circumscribed rectangle, circumscribed rectangle, circumscribed circle), as in the second embodiment. Is set as the luminance conversion target area.
When the luminance conversion target area setting processing unit 3h sets N luminance conversion target areas (1) to (N), the luminance L of the pixels existing in the luminance conversion target areas (1) to (N) is set. (I, j) is stored in the memories 4 a-1 to 4 a -N of the luminance conversion unit 4.
Needless to say, the luminance L (i, j) of the pixel stored in the memories 4a-1 to 4a-N is not only information indicating the luminance of the pixel, but also the original HDR in which the pixel exists. Information indicating the position (i, j) in the image is also included.

  As is apparent from the above, even when the luminance conversion target area setting unit 3 includes the high luminance small area removing unit 3f, the clustering unit 3i, and the luminance conversion target area setting processing unit 3h, the same as in the first embodiment. In addition, it is possible to perform luminance conversion for each region having similar luminance characteristics, so that the observation target object in each region can be displayed in an easily understandable manner.

Embodiment 4 FIG.
In the first embodiment, the luminance conversion unit 4 exists in the luminance conversion target region (n) for each of the luminance conversion target regions (1) to (N) set by the luminance conversion target region setting unit 3. The luminance L (i, j) of the pixel existing in the luminance conversion target region (n) is removed by linear luminance compression after the noise is removed, and the luminance cL (i, j, in the luminance conversion range S (n) is removed. j), the pixel luminance L (i, j) is determined in consideration of the bias of the luminance L (i, j) of the pixel existing in the luminance conversion target area (n). You may make it convert.
Specifically, it is as follows.

FIG. 14 is an explanatory diagram showing the luminance conversion processing by the luminance conversion unit 4.
FIG. 14 shows the score for the luminance of the histogram in the dotted line range, and the luminance conversion unit 4 determines the display range and the target luminance range.
For example, as shown in FIG. 14, the luminance range of interest includes a peak (maximum) maximum point and is determined as a left and right range until the gradient of the histogram becomes gentle.

There are four possible methods for determining the display range, and the luminance conversion unit 4 determines the display range according to one of the determination methods.
(1) A method for setting the display range below a certain threshold value (2) A method for setting a pixel in the range from low luminance pixels to 90% of the total number of pixels as a display range (3) High luminance small region removal unit 3f (4) A luminance range that includes a maximum point as in the histogram on the left side of FIG. 14 and has an equal number of pixels on the left and right, and , The number of pixels in the luminance range is 1 / number of pixels in the luminance conversion target area.
How to set the range of 3 as the display range

  When the luminance conversion unit 4 determines the display range and the attention luminance range, 85 gradations (grayscales from 85 to 169) centering on the attention luminance range among 255 gradations as shown in the histogram on the right side of FIG. The luminance smaller than the luminance range of interest is converted into gradations from 0 to 84. In addition, luminance that is larger than the luminance range of interest is converted to a gradation of 170 to 254.

  As apparent from the above, the luminance conversion unit 4 considers the bias of the luminance L (i, j) of the pixel existing in the luminance conversion target region (n), and the luminance L (i, j) of the pixel. However, as in the first embodiment, optimum luminance conversion can be realized for each of the luminance conversion target areas (1) to (N), and the luminance conversion target area (1 ) To (N), the observation object can be displayed in an easily understandable manner.

Embodiment 5 FIG.
In the first embodiment, the area synthesis unit 5 reads the luminance-converted images in the luminance conversion target areas (1) to (N) stored in the memories 4c-1 to 4c-N of the luminance conversion unit 4. In the above description, the luminance L (i, j) of the pixels in the luminance conversion target areas (1) to (N) displayed on the display is replaced with the luminance cL (i, j) of the image after luminance conversion. When the region composition unit 5 replaces the luminance cL (i, j) of the image after luminance conversion, the average luminance L _ave in the N luminance conversion target regions (1) to (N) is calculated, and the average luminance is calculated. luminance cL (i, j) which has been converted by the brightness conversion portion 4 in response to the L _ave may be shifted gradations.
Specifically, it is as follows.

Here, for convenience of explanation, it is assumed that three luminance conversion target areas (1) to (3) are set.
The region synthesis unit 5 reads an image of the luminance conversion target region (1) and calculates an average luminance L _{ave1 of the} pixels constituting the image.
Similarly, the region synthesis unit 5 calculates average luminances L _ave2 and L _ave3 of the pixels constituting the images of the luminance conversion target regions (2) and (3).

After calculating the average luminances L _ave1 , L _ave2 , and L _ave3 in the luminance conversion target regions (1) to (3), the region synthesis unit 5 compares these average luminances with each other to determine the brightness order. To do.
Here, for convenience of explanation, the brightest average luminance L _ave2 of luminance conversion target area (2), second brightest, the average luminance L _ave1 of luminance transformation target area (1) in the luminance transformation target area (3) in It is _assumed that the average luminance _Lave3 is the darkest.
L _ave2 > L _ave1 > L _ave3

When the region composition unit 5 determines the order of brightness, the region-converted images (255 gradation images) in the three luminance conversion target regions (1) to (3) are converted to 80% gradation (however, , Integer).
That is, the brightness-converted image in the brightest brightness conversion target area (2) is shifted to 0.2 × 255 to 1.0 × 255 gradations.
Next, the brightness-converted image in the bright brightness conversion target area (1) is shifted to the gradation of 0.1 × 255 to 0.9 × 255.
The brightness-converted image in the darkest brightness conversion target area (3) is shifted to 0 × 255 to 0.8 gradation.
However, the value is rounded so that the luminance becomes an integer.

When the gradation of the luminance cL (i, j) converted by the luminance conversion unit 4 is shifted according to the average luminance L _ave in the luminance conversion target regions (1) to (3), the region synthesis unit 5 shifts to the display. The luminance L (i, j) of the displayed pixels in the luminance conversion target areas (1) to (N) is replaced with the luminance cL (i, j) after the gradation shift.

As apparent from the above, according to the fifth embodiment, when there are a plurality of luminance conversion target areas set by the luminance conversion target area setting unit 3, the average luminance L _ave in the plurality of luminance conversion target areas is calculated. Since the gradation of the luminance cL (i, j) calculated and converted by the luminance conversion unit 4 is shifted according to the average luminance L _ave , the observation objects in the plurality of luminance conversion target regions are displayed. There is an effect that can be easily distinguished.

  DESCRIPTION OF SYMBOLS 1 HDR image storage part, 2 conversion area range designation | designated part (conversion area range designation | designated means), 3 brightness | luminance conversion object area | region setting part (brightness conversion object area | region setting means), 3a smoothing process part, 3b climax surface process part, 3c Differential operator processing unit, 3d group classification processing unit, 3e luminance conversion target region setting processing unit, 3f high luminance small region removing unit, 3g clustering unit, 3h luminance conversion target region setting processing unit, 3i clustering unit, 4 luminance conversion unit ( (Brightness conversion means), 4a-1 to 4a-N memory, 4b-1 to 4b-N, luminance conversion processing section, 4c-1 to 4c-N memory, 5 area composition section (area composition means), 6 composite image storage section , 7 Stereoscopic display unit (stereoscopic display means).

Claims (10)

  Among pixels constituting an image having a luminance gradation exceeding the visible range, pixels whose luminance is close to that of adjacent pixels are classified into the same group, and pixels belonging to the same group exist. For each luminance conversion target area set by the luminance conversion target area setting means for setting the area as the luminance conversion target area and the luminance conversion target area setting means, noise existing in the luminance conversion target area is removed. A luminance conversion means for converting the luminance of a pixel existing in the luminance conversion target area into a luminance within a visible range, and one or more luminance conversion target areas in which the luminance of the pixel is converted by the luminance conversion means An image display device comprising: a region synthesizing unit that synthesizes the images and generates a synthesized image.   A conversion area range specifying unit is provided that specifies a range of a region to be subjected to luminance conversion in an image having a luminance gradation exceeding the visible range and outputs the image in the region to a luminance conversion target region setting unit. The image display device according to claim 1.   3. The image display device according to claim 1, further comprising a stereoscopic display means for stereoscopically displaying the luminance of the pixels constituting the synthesized image generated by the area synthesizing means. The luminance conversion target region setting means includes a ridge surface processing unit that performs a tangential surface processing on pixels constituting an image having a luminance gradation exceeding the visible range;
A differential operator processing unit that calculates the gradient of the pixel value in the image after the tangential surface processing by performing differential operator processing on the pixels that have been subjected to the tangential surface processing by the tangential surface processing unit;
A group classification processing unit that classifies a pixel group in which pixels having a gradient of a pixel value calculated by the differential operator processing unit below a threshold among the pixels constituting the image into one group; and
The luminance conversion target region setting processing unit configured to set a circumscribed region including pixels belonging to the group classified by the group classification processing unit as a luminance conversion target region. 4. The image display device according to claim 1.   The group classification processing unit temporarily sets a threshold value to be compared with the gradient of the pixel value calculated by the differential operator processing unit, and immediately before the number of pixels belonging to one group rapidly increases while sequentially decreasing the threshold value. 5. The image display apparatus according to claim 4, wherein a pixel group in which pixels having a gradient of pixel values calculated by the differential operator processing unit is equal to or less than the threshold value is grouped into one group. .   The luminance conversion means determines a pixel whose luminance is a predetermined value or more higher than the luminance of surrounding pixels among the pixels existing in the luminance conversion target region as noise, and uses a low-pass filter to reduce the noise. The image display device according to claim 1, wherein the image display device is removed.   The luminance conversion means removes noise existing in the luminance conversion target region, and then converts the luminance of the pixels existing in the luminance conversion target region into luminance within the luminance conversion range by linear luminance compression. The image display device according to claim 6.   The luminance conversion means, after removing the noise present in the luminance conversion target region, calculates the average value and the variance value of the luminance of the pixels existing in the luminance conversion target region, the above average value and The luminance conversion range is determined from the variance value within the visible range, and the luminance of the pixels existing in the luminance conversion target region is changed to the luminance within the luminance conversion range using the average value and the variance value. 7. The image display device according to claim 6, wherein the image display device performs conversion.   The area synthesis means calculates the brightness of the pixels existing in the brightness conversion target area set by the brightness conversion target area setting means among the brightness of the pixels constituting the image having the brightness gradation exceeding the visible range. 9. The image display device according to claim 1, wherein the image display device is replaced with the luminance converted by the luminance conversion means.   When there are a plurality of brightness conversion target areas set by the brightness conversion target area setting means, the area synthesis means calculates the average brightness in each of the plurality of brightness conversion target areas, and the brightness conversion means according to the average brightness. The image display apparatus according to claim 9, wherein the gradation of the converted luminance is shifted.

Images