JP2004220438A - Image processing program and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform image processing which correctly reflects user intention when generating a completed image by range composition of main data and highlight data. <P>SOLUTION: CCR-RAW data recorded in a CCD-RAW recording mode is replayed after various image processes are applied by CCD-RAW application software. The CCD-RAW application software performs range composition of the main data and the highlight data and then applies highlight compression process for compressing mainly tonal width of high luminance region to keep the composition image within a predetermined tonal width. Processing conditions of the highlight compression process can be adjusted by a user and the CCD-RAW application software changes compression parameters according to specification by the user. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
According to the present invention, each pixel data includes main image data expressing gradations from a low luminance region to a medium luminance region, and highlight image data expressing gradations of a high luminance region that cannot be expressed by the main image data. The present invention relates to an image processing method and program for configured image data.
[0002]
[Prior art]
An image sensor such as a CCD image sensor generally has a narrow dynamic range (ratio between the reproducible minimum value and maximum value) that expresses the reproducibility of the luminance signal compared to a photographic film. In such a case, the high luminance area may be skipped or the low luminance area may be blackened. Therefore, by combining the two types of data, the main data that mainly expresses the gradation in the low to medium luminance range and the highlight data that expresses the gradation in the high luminance range that cannot be expressed with this main data, A method for obtaining a completed image having a wide dynamic range has been proposed.
[0003]
The two types of data capture methods include a method of obtaining main data and highlight data by photographing the same scene a plurality of times with different exposure amounts (see, for example, Patent Document 1 below), or FIG. 9A. As shown, one pixel 71 uses a special CCD image sensor 70 composed of a main pixel 71a and a sub-pixel 71b having a smaller light receiving area than the main pixel 71a. For example, there is a method of fetching main data from the sub-pixel 71b having low sensitivity. As shown in the graph of FIG. 9B, since the main pixel 71a has high sensitivity, it is saturated with a smaller amount of incident light than the sub-pixel 71b. On the other hand, since the subpixel 71b has low sensitivity, it is saturated with a larger amount of incident light than the main pixel 71a. Thereby, the main image data can be extracted from the main pixel 71a, and the highlight data can be extracted from the sub-pixel 71b.
[0004]
The main data and highlight data obtained by such a method are converted into data having a gradation width of 10 bits (1024 gradations), for example. By synthesizing the range of the main data and the highlight data, a synthesized image having a wider dynamic range than the dynamic range of only the main data can be obtained. However, the gradation width of this composite image has a gradation width that is double that of each data by combining two types of data. Since this gradation width is too wide when the image is used for monitor display, it is necessary to accommodate this within a predetermined gradation width (for example, a 10-bit gradation width). Therefore, highlight compression processing is mainly performed on the synthesized image to compress the gradation width in the high luminance region. This highlight compression is executed in accordance with a compression parameter, which is determined in advance as processing parameters for how much luminance range to compress.
[0005]
[Patent Document 1]
JP-A-2002-271686 [0006]
[Problems to be solved by the invention]
However, when performing the above-described highlight compression, the image quality of the completed image varies greatly depending on what compression parameter is used. However, in the conventional image processing method, the compression parameter is determined in advance. The user could not change this. For this reason, it is difficult to obtain the image quality intended by the user.
[0007]
In addition, in order to more accurately reflect the user's intention in the completed image, brightness correction (sensitization and desensitization correction, or digital exposure correction, etc.) that corrects the brightness of the entire screen is performed together with highlight compression processing. It is better to be able to do it. However, since highlight compression and brightness correction are common in that the input / output ratio of the luminance signal is changed, they are closely related. For this reason, when performing brightness correction together with highlight compression, if the brightness correction parameter (EV value) is set to a positive value and sensitized, whiteout in a high luminance region occurs. If the effect of the range composition performed to reproduce the gradation in the high luminance region is not obtained, or if the EV value is set to minus and desensitized, the high luminance region becomes dark. Occurs and the contrast is lowered. For this reason, even if the degree of freedom of adjustment by the user is increased in order to accurately reflect the user's intention, there is a problem that it becomes very difficult to use it.
[0008]
The present invention can provide an image processing method and program capable of accurately reflecting a user's intention on a completed image obtained by range synthesis and highlight compression.
[0009]
It is another object of the present invention to provide an image processing method and program that can be easily adjusted while increasing the degree of freedom of adjustment of processing conditions by a user when performing image processing including range synthesis and highlight compression. To do.
[0010]
[Means for Solving the Problems]
In order to achieve the above object, an image processing method and program according to the present invention provide main data representing gradations from a low luminance region to a medium luminance region in order to obtain an image having a wide dynamic range, and the main data. The step of generating a composite image having a wide dynamic range by performing range composition processing based on two types of data, ie, highlight data representing gradations in a high luminance region that cannot be represented by A step of generating a finished image by executing a highlight compression process for compressing a gradation width of a high-luminance region and keeping a total gradation width of the composite image within a predetermined gradation width; and processing conditions of the highlight compression process. And a step of changing according to the user's specification.
[0011]
Furthermore, when the processing conditions for highlight compression processing are changed, a preview image to which the processing conditions are applied may be displayed on the monitor.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
A digital camera 10 shown in FIG. 1 includes the above-described special CCD image sensor 70 (see FIG. 9) as a CCD image sensor. The CCD image sensor 70 takes in each of main data and highlight data. The digital camera 10 also has a CCD-RAW recording mode. In the CCD-RAW recording mode, CCD-RAW data, which is data immediately after the analog imaging signal output from the CCD image sensor 70 is converted into digital data by the A / D converter, is not subjected to image processing inside the camera. It is recorded in the storage medium 11 as it is. As for CCD-RAW data, each of main data and highlight data has a gradation width of 10 bits, for example.
[0013]
Since the CCD-RAW data cannot be reproduced by a general-purpose image processing program, the personal computer 12 in which a dedicated image processing program is installed performs gain correction and gamma correction as well as sharpness according to user preferences. Image processing such as correction, saturation correction, and brightness correction is performed and reproduced. The synthesis process of the main data and the highlight data is executed by the image processing program. The personal computer 12 includes a computer main body 13, a monitor 14, and input devices 18 such as a keyboard 16 and a mouse 17.
[0014]
As shown in FIG. 2, the computer main body 13 includes a CPU 21, a RAM 22, a data storage 23 including a hard disk drive, and a communication I / F 24. The communication I / F 24 is an image input unit that is connected to the digital camera 10 via a communication cable and captures an image file from the storage medium 11 set in the camera body into the computer body 13. The RAM 22 is a working memory when the CPU 21 executes processing.
[0015]
In the data storage 23, a CCD-RAW application 26 that is an image processing program is installed. The completed image generated by the CCD-RAW application 26 is output to the data storage 23. The CCD-RAW application 26 includes a program main body 27, an LUT 28, operation screen data 30, and the like.
[0016]
The CCD-RAW application 26 causes the CPU 21 to execute processing condition settings for setting image processing conditions in accordance with a user's specification and image processing steps for executing various image processing in accordance with the set processing conditions.
[0017]
The LUT 28 stores various image processing correction parameters in a table format. The LUT 28 includes a brightness correction LUT 28a storing brightness correction parameters, and a compression LUT 28b storing compression parameters for performing highlight compression processing for compressing a range-combined composite image to a predetermined gradation width. Etc. are included. These are loaded into the RAM 22 and used as needed. As the correction parameter, a predetermined predetermined value set in advance is set, and the correction parameter is stored in each LUT. This default correction parameter can be adjusted by user designation, and each LUT is changed accordingly.
[0018]
FIG. 3 is an explanatory diagram of a synthesis process including a range synthesis process of main data and highlight data and a highlight compression process for a synthesized image generated by the synthesis. The main data M captured from the CCD image sensor 70 has a dynamic range DR1 because the exposure amount is saturated at Q1. On the other hand, the highlight data H is saturated at an exposure amount Q2 larger than the exposure amount Q1. With this highlight data H, gradations in a high luminance region that cannot be represented by the main data M are represented. A range composition process is executed based on the main data M and the highlight data H, and composite image data (composite data) having a wider dynamic range DR2 than the dynamic range DR1 is obtained.
[0019]
Since the composite data G has a gradation width twice that of the main data M, it is not suitable for general use as it is. In view of this, highlight compression is performed by compressing a predetermined gradation width (for example, 1024) while maintaining a wide dynamic range DR2 by mainly compressing a high luminance area of the gradation width of the composite data. Thereby, the compressed data C is obtained. The compressed data C is subjected to other correction processes such as white balance correction and saturation correction, and is output as a completed image.
[0020]
FIG. 4 shows an example of the compression parameter P0 used in highlight compression. The gain is “0.50” so that the maximum value of the signal level (gradation value) is changed from “2046” to “1023”. Further, in the portion from “0” to “1023” corresponding to the low luminance area to the middle luminance area of the composite image, the gain is “0” to “0” so that the gradation value is compressed by about 20 to 30%. 0.78 ". The compression parameter P0 is set as a default value, but this compression parameter greatly affects the image quality. Therefore, the user can adjust the compression parameter so that the image quality according to the user's preference can be obtained.
[0021]
Also, depending on the user, when performing synthesis (range synthesis + highlight compression), brightness may be adjusted together with synthesis. FIG. 5 is an explanatory diagram of compressed data when brightness correction is performed simultaneously with synthesis. FIG. 5A shows compressed data C0 + 1EV when the sensitization correction of + 1EV is performed by brightness correction on the compressed data C0 that has been subjected to highlight compression using the standard compression parameter P0. FIG. 5C shows compressed data C0-1EV when the desensitization correction of -1EV is performed on the compressed data C0 by brightness correction. As is apparent from these graphs, when the brightness correction is performed using the standard compression parameter P0, the output signal level in the high luminance region exceeds the maximum gradation value “1023” having a 10-bit width during sensitization. The so-called whiteout occurs, and the effect of reproducing the gradation in the high luminance range by performing range synthesis cannot be obtained. On the other hand, at the time of desensitization, a white sink occurs in which the high luminance area is entirely sinked.
[0022]
Therefore, in the present invention, when both synthesis and brightness correction are performed, the compression parameter is determined according to the brightness adjustment level designated by the user, and whitening at the time of sensitization and whitening at the time of desensitization are determined. We are trying to prevent sinking. FIG. 5B shows compressed data C1 that has been subjected to highlight compression using the compression parameter P1 for sensitization correction, and compressed data C1 + 1EV that has been sensitized by + 1EV with respect to this compressed data C1. . FIG. 5D shows compressed data C2 that has been subjected to highlight compression using the compression parameter P2 for desensitization correction, and compressed data C2-1EV that has undergone -1 EV desensitization to this compressed data C2. It shows. FIG. 6 shows examples of the compression parameter P1 for sensitization correction and the compression parameter P2 for desensitization correction.
[0023]
As can be seen from the table of FIG. 6 and the graph of FIG. 5 (B), in the compression parameter P1 for sensitization, when the maximum value of the output signal level is sensitized by brightness correction, the maximum gradation value “ The gradation value gain is set to “0.25” so as to be “1023”. When range synthesis and highlight compression are executed using the compression parameter P1, compressed data C1 is obtained. Then, + 1EV sensitization correction is performed on the compressed data C1 to obtain compressed data C1 + 1EV. In the compressed data C1 + 1EV, in the low luminance region to the middle luminance region, the brightness is increased as compared with the compressed data C0 using the standard parameters, and there is no whiteout in the high luminance region.
[0024]
On the other hand, as can be seen from the graph of FIG. 5D, in the desensitization compression parameter P2, when the maximum value of the output signal level is desensitized by the brightness correction, the maximum gradation value is “1023”. As described above, the gain of the gradation value is set to “1”. When range synthesis and highlight compression are executed using the compression parameter P2, compressed data C2 is obtained. Then, -1EV desensitization correction is performed on the compressed data C2, and compressed data C2-1EV is obtained. In the compressed data C2-1EV, in the low luminance region to the medium luminance region, the desensitization is performed as compared with the compressed data C0 using the standard parameters, and the white sink in the high luminance region is eliminated.
[0025]
In FIG. 6, one parameter for sensitization and one for desensitization for performing ± 1 EV correction is shown as one parameter for brightness correction. However, other brightness correction levels (adjustment values) are available. Since various levels are assumed, a large number of compression parameters corresponding to the levels are prepared and stored in the brightness LUT 28a. Of course, the parameters are stored in the LUT format for correction, but the correction may be made by a function equation or the like.
[0026]
As shown in FIG. 7, the CCD-RAW application 26 employs a GUI (Graphical User Interface) that accepts various operation instructions through an operation screen 31 displayed on the monitor 14. The operation screen 31 includes an image display area 32 and an adjustment unit display area 33. In the image display area 32, a preview image 34 of the selected image file is displayed. The CCD-RAW application 26 is activated with a desired image file selected. When the CCD-RAW application 26 is activated, the CPU 21 executes each processing step included in the application 26. The CPU 21 sets a predetermined value for the processing condition (correction parameter) immediately after the CCD-RAW application 26 is activated, and displays the operation screen 31 on the monitor 14.
[0027]
The adjustment unit display area 33 includes adjustment units 36, 37, 38, 39, and 40 for designating processing conditions of white balance correction, saturation correction, sharpness correction, highlight compression, and brightness correction. ing. Each of the adjustment units 36 to 40 is provided with a scale and sliders 36a to 40a that move on the scale. The correction level of each correction process is adjusted by dragging the pointer 41 to the sliders 36a to 40a and dragging them in the scale direction. When the slider 39a is moved from the initial position (center) in the left-right direction ("weak" or "strong" direction), the default value of the compression parameter for highlight compression is changed. Further, when the slider 39a is not moved from the initial position (center), the default value of the compression parameter is not changed, but even in that case, the brightness correction slider 40a is moved from the initial position (center). When the brightness correction level is adjusted, the compression parameter for sensitization or desensitization is changed as described above according to the correction level.
[0028]
When the processing condition is changed through each of the adjustment units 36 to 40, the processing condition is applied in real time, and the preview image 34 is updated. By doing so, the result of the processing condition specified by the user can be confirmed in advance. Also, below the image display area 32, a determination button 43 for confirming the processing conditions and a cancel button 44 for returning the processing conditions to the default values are provided. When the determination button 43 is clicked with the mouse 17, the correction level designated by the user is confirmed. When the processing conditions are determined, the CPU 21 performs image processing on the main image data based on the determined processing conditions, and outputs a completed image to the data storage 23.
[0029]
Hereinafter, the operation of the above configuration will be described with reference to the flowchart of FIG. When an image file recorded in the CCD-RAW recording mode of the digital camera 10 is selected and an activation instruction is given, the CCD-RAW application 26 is activated. When the CCD-RAW application 26 is activated, the CPU 21 executes each processing step of the CCD-RAW application 26. The CPU 21 reads the LUT 28 from the data storage 23 immediately after startup, sets the default value of the processing condition, and displays the operation screen 31 on the monitor 14. The preview image 34 is generated based on the default value of the set processing condition.
[0030]
While viewing the preview image 34, the user operates the adjustment units 36 to 40 through the mouse 17 to designate desired processing conditions. When the processing condition is adjusted, the CPU 21 changes the default value of the processing condition according to the user's designation, and updates the preview image 34 based on the changed correction parameter. Since the processing conditions are reflected in the preview image 34 in this way, adjustment is easy. If the compression processing conditions are adjusted when adjusting the processing conditions, the CPU 21 changes the compression parameters accordingly. As described above, since the compression parameter can be changed in accordance with the user's designation, the user's intention can be more accurately reflected in the completed image.
[0031]
When the brightness adjustment is performed, the compression parameter is determined according to the brightness adjustment value designated by the user. For this reason, whiteout does not occur in the high luminance region during sensitization, and white sink does not occur during desensitization. In this way, since the compression parameter is determined according to the brightness correction level specified by the user, even if the user intuitively adjusts the brightness, the image quality will not deteriorate. Therefore, the processing conditions can be easily adjusted.
[0032]
When the adjustment is completed, the determination button 43 is clicked with the mouse 17. As a result, the adjusted processing conditions are determined. The CPU 21 executes image processing based on the determined processing conditions. The completed image generated by executing the image processing is output to the data storage 23.
[0033]
In the above embodiment, the image processing program is installed in a computer. However, the image processing method of the present invention is not limited to this embodiment. For example, the image processing program can be realized by a dedicated image processing apparatus, The present invention can be applied to various forms such as a processing program incorporated in a digital camera body and realized by a digital camera.
[0034]
【The invention's effect】
As described above in detail, the present invention provides main data that expresses gradations from a low luminance region to a medium luminance region, and highlight data that expresses gradations in a high luminance region that cannot be expressed by the main data. When image processing for generating a composite image having a wide dynamic range based on two types of data is performed, the gradation width of mainly the high luminance region of the generated composite image is compressed, and the total gradation width of the composite image is reduced. Since it includes a step of generating a completed image by executing a highlight compression process that fits within a predetermined gradation width, and a step of changing the processing conditions of the highlight compression process according to a user's specification. The user's intention can be accurately reflected in the image.
[0035]
Furthermore, when the processing conditions of the highlight compression process are changed, the preview image to which the processing conditions are applied is displayed on the monitor, so that it is easy to adjust the processing conditions.
[0036]
Further, when performing brightness correction processing in addition to the highlight compression processing, the processing conditions for the highlight compression processing are determined according to the processing conditions for the brightness correction processing. While increasing the degree of freedom in adjusting the processing conditions, the adjustment can be simplified.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram of a method for reproducing CCD-RAW data.
FIG. 2 is a configuration diagram of a CCD-RAW application and a personal computer on which it is installed.
FIG. 3 is an explanatory diagram of an image synthesis process of main data and highlight data.
FIG. 4 is an example of compression parameters.
FIG. 5 is an explanatory diagram of compressed data obtained by performing brightness correction together with range synthesis.
FIG. 6 is an example of compression parameters when brightness correction is performed together with range synthesis.
FIG. 7 is an example of an operation screen of a CCD-RAW application.
FIG. 8 is a flowchart showing a procedure for generating a completed image from CCD-RAW data using a CCD-RAW application.
FIG. 9 is an explanatory diagram of a CCD image sensor capable of generating an image having a wide dynamic range.
[Explanation of symbols]
10 Digital Camera 11 Storage Media 12 Personal Computer 26 CCD-RAW Application 28 LUT
31 Operation screen

Claims (6)

In order to obtain an image having a wide dynamic range, main data representing gradations from a low luminance region to a medium luminance region and highlight data representing gradations in a high luminance region that cannot be represented by the main data are 2 Performing a range synthesis process based on the type of data to generate a composite image having a wide dynamic range;
A step of generating a completed image by executing a highlight compression process for compressing a gradation width of a high-luminance region of the generated composite image and keeping a total gradation width of the composite image within a predetermined gradation width;
An image processing program for causing a computer to execute processing, comprising: changing processing conditions of the highlight compression processing in accordance with user designation. 2. The image processing program according to claim 1, wherein when a processing condition for highlight compression processing is changed, a preview image to which the processing condition is applied is displayed on a monitor. 2. The brightness compression process when the brightness correction process is performed in addition to the highlight compression process, the process condition of the highlight compression process is determined according to the process condition of the brightness correction process. 2. The image processing program according to 2. In order to obtain an image having a wide dynamic range, main data representing gradations from a low luminance region to a medium luminance region and highlight data representing gradations in a high luminance region that cannot be represented by the main data are 2 Performing a range synthesis process based on the type of data to generate a composite image having a wide dynamic range;
A step of generating a completed image by executing a highlight compression process for compressing a gradation width of a high-luminance region of the generated composite image and keeping a total gradation width of the composite image within a predetermined gradation width;
And a step of changing processing conditions of the highlight compression processing in accordance with user designation. 5. The image processing method according to claim 4, further comprising: when a processing condition for highlight compression processing is changed, a preview image to which the processing condition is applied is displayed on a monitor. 5. When performing brightness correction processing in addition to the highlight compression processing, processing conditions for the highlight compression processing are determined according to processing conditions for brightness correction processing. 5. The image processing method according to 5.

Images