JP2009282860A - Developing processor and development processing method for undeveloped image data, and computer program for development processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve convenience of adjustment of a developing parameter. <P>SOLUTION: This developing processor generates already developed image data by applying development processing to undeveloped image data generated by a digital camera. A first parameter related to the developing parameter used for the development processing is determined by a developing parameter determination part. The developing processor displays a parameter change screen, and displays the first parameter determined by the developing parameter determination part and a second parameter to be changed by a user on the parameter change screen. The development processing is performed based on at least the second parameter. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for developing undeveloped image data generated by a digital camera.

  In recent years, digital still cameras record RAW image data on a memory card. The RAW image data recorded on the memory card in this manner is converted into image data such as JPEG format or TIFF format, image data for printing (developed image data) or the like that is normally used by a development processing application of a personal computer. Converted.

  Processing for converting RAW image data into image data in a format that is normally used (referred to as “development processing”. At this time, since RAW image data is data before development processing, it is also referred to as “undeveloped image data”. ), The image quality of the generated developed image data varies greatly depending on the processing parameters (development parameters) during the development processing. Therefore, the development processing application or the like generally has a parameter adjustment function so that the user can adjust these parameters.

JP 2005-202749 A JP 2007-124599 A

  However, a user interface used when a user changes development parameters is generally created for a person who is familiar with development processing. Therefore, the convenience of adjusting development parameters is not always sufficient for general users. Specifically, it is difficult for a general user to return the development parameters to the state before the change when the development parameter change results are not preferable or when the development parameters are changed by mistake. .

  The present invention has been made to solve the above-described problems in the prior art, and an object thereof is to improve the convenience of adjusting development parameters.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

[Application Example 1]
A development processing device for developing undeveloped image data generated by a digital camera,
A development processing unit that generates developed image data in a predetermined format from the undeveloped image data by performing development processing on the undeveloped image data;
A development parameter determination unit for determining a first parameter related to the development parameter used in the development processing;
A change screen display for displaying a parameter change screen;
With
The change screen display unit displays the first parameter determined by the development parameter determination unit and the second parameter changed by the user on the parameter change screen,
The development processing unit performs development processing based on at least the second parameter.

  According to this application example, since the first parameter and the second parameter are displayed on the parameter change screen, the user can intuitively understand the parameter item that the user has changed. Further, for a parameter to be changed, the parameter can be changed with reference to both the first parameter and the second parameter displayed on the parameter change screen. For this reason, it is easier to return to the state before the change after changing the parameter once, and the convenience of adjusting the development parameter is improved.

[Application Example 2]
A development processing apparatus according to Application Example 1,
The first parameter can be expressed as a one-dimensional numerical value;
The change screen display section
Displaying the first parameter as a first marker image indicating a position on a number line corresponding to the first parameter displayed on the parameter change screen, which is set to be unmovable by the user;
A development processing apparatus that displays the second parameter as a second marker image that indicates a position on the number line that is set to be movable by a user.

  According to this application example, the user can return the changed parameter to the state before the change by moving the second marker image to a position corresponding to the first marker image. Therefore, it becomes easier to return the development parameters to the state before the change, and the convenience of the development parameter adjustment is improved.

[Application Example 3]
A development processing apparatus according to Application Example 2,
The second marker image is displayed in different colors when the first parameter and the second parameter are the same, and when the first parameter and the second parameter are different. Development processing device.

  According to this application example, the color of the second marker image is displayed differently depending on whether or not the first parameter and the second parameter are the same. Therefore, the user can more easily grasp the parameters that the user has changed.

[Application Example 4]
A development processing apparatus according to Application Example 3,
The color of the second marker image changes according to the difference between the first parameter and the second parameter.

  According to this application example, the color of the second marker image changes according to the difference between the first parameter and the second parameter. Therefore, the user can more easily grasp the parameter change amount based on the color of the second marker image.

[Application Example 5]
A development processing apparatus according to Application Examples 1 to 4,
The change screen display section
The first parameter is displayed on the parameter change screen as a character string that cannot be rewritten by the user, and the second parameter is displayed on the parameter change screen as a character string that can be rewritten by the user. Development processing device.

  According to this application example, the user can change the second parameter by directly inputting characters. Therefore, it becomes easier to change the second parameter to a desired value.

  The present invention can be realized in various modes. For example, the development processing apparatus and the development processing method, the image output apparatus and the image output method using the development processing apparatus or the development processing method, and the like. The present invention can be realized in the form of a computer program for realizing the function of the method or apparatus, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Example:
A1. Photo viewer configuration:
A2. Start development:
A3. RAW image generation:
A4. Automatic adjustment of development parameters:
A5. Display processing:
A6. Preview screen display:
A7. Main development:
B. Modified preview screen:
C. Variations:

A1. Photo viewer configuration:
FIG. 1 is a schematic configuration diagram schematically showing a configuration of an image data generation system 10 to which an embodiment of the present invention is applied. The image data generation system 10 includes a digital still camera 100 and a photo viewer 200. Each of the digital still camera 100 and the photo viewer 200 includes a memory card slot for storing a memory card MC indicated by a one-dot chain line.

  The digital still camera 100 generates an image file in a predetermined format from the captured image. The generated image file is stored in the memory card MC. An image photographed by the digital still camera 100 can be confirmed on the photo viewer 200 by inserting a memory card MC storing an image file into the photo viewer 200.

  FIG. 2 is an explanatory diagram illustrating an example of a data format of an image file generated by the digital still camera 100 and stored in the memory card MC. The image file shown in FIG. 2 is an image file in a format called a RAW image file, in which RAW data representing the output value of the image sensor (CCD, CMOS, etc.) of the digital still camera 100 is stored. This RAW image file is created in a data format similar to the Exif (Exchangeable Image File Format) format, and includes a header portion and a data portion as shown in the figure.

  In the header section, additional information such as the manufacturer name and model number of the camera that created the RAW image file, imaging conditions, and device characteristics are described. Note that the shooting conditions include setting conditions at the time of shooting such as shutter speed, aperture value, white balance setting value, and scene setting. Further, the device characteristics include various parameters representing the characteristics of the device of the digital still camera 100 such as an offset value (described later) with respect to the gradation value.

  The data portion includes RAW data and screen nail image data generated at the time of shooting. Screen nail image data is image data (developed image data) obtained by performing development processing (image generation processing) on RAW data. RAW data and screen nail image data are recorded at the same time as shooting, and are data representing the same subject (captured image). The screen nail image data is, for example, image data in JPEG format that is used when a captured image is simply displayed on a display panel provided in the camera and is developed in the camera.

  The data format of the RAW image file, RAW data, and screen nail image data differs depending on the manufacturer and model of the digital still camera 100 (FIG. 1). The screen nail image data has, for example, the same number of pixels as the RAW data, a smaller number of pixels than the RAW data obtained by reducing the RAW data for display, or a pixel of about VGA (640 × 480 pixels). Set to a number.

  FIG. 3 is a schematic configuration diagram schematically showing the configuration of the photo viewer 200. The photo viewer 200 includes a CPU 210, a ROM 220, a RAM 230, a display controller 240, an input port 250, a memory card interface 260, and a hard disk drive (HDD) 270. Note that the photo viewer 200 has an external device interface (not shown) for connecting to a personal computer or a printer. However, the connection with the external device is not directly related to the present invention, and the description thereof will be given here. Omitted.

  A liquid crystal display 242 is connected to the display controller 240. The display controller 240 displays various images on the liquid crystal display 242 by controlling the liquid crystal display 242 based on the image data supplied from the CPU 210. Note that the display controller 240 of this embodiment receives image data in YUV format (ITU-R BT.601, ITU-R BT.656, etc.) standardized by ITU-R (International Telecommunication Union Wireless Communication Division). Accept.

  An operation button 252 provided on the photo viewer 200 is connected to the input port 250. The input port 250 supplies various button operation states included in the operation button 252 to the CPU 210. Thereby, CPU210 can acquire the various instruction | indication given from a user. The operation buttons 252 include, for example, a reduction button BZD, an enlargement button BZU, a direction button BDR, an OK button BOK, and a cancel button BRT for canceling the previous operation.

  A memory card slot 262 is connected to the memory card interface 260. The memory card interface 260 is an interface for mediating data exchange between the memory card MC inserted into the memory card slot 262 and the CPU 210.

  The CPU 210 executes various programs stored in the ROM 220, RAM 230, or HDD 270 to control each unit 220, 230, 240, 250, 260, 270 provided in the photo viewer 200, thereby various functions provided in the photo viewer 200. To realize. RAM 230 and HDD 270 store temporary data when CPU 210 executes a program. In addition to these programs and temporary data, the HDD 270 stores image data read from the memory card MC.

A2. Start development:
FIG. 4 is a flowchart showing the flow of development processing executed by the CPU 210. The development process shown in FIG. 4 is executed when an instruction to execute the development process is given to the photo viewer 200 from the user. When the CPU 210 executes this development processing, a function as a development processing apparatus included in the photo viewer 200 is realized.

  In step S102, the CPU 210 acquires an instruction (development designation) for designating the development content given by the user. As development contents, a RAW image file in which RAW data to be developed is stored and a parameter (development parameter) adjustment mode for development processing are designated. Specifically, the CPU 210 generates a menu screen for acquiring these designations, and supplies the generated menu screen to the display controller 240 to display the menu screen on the liquid crystal display 242. Then, by acquiring the operation state of the operation button 252 via the input port 250, the development designation is acquired.

  FIG. 5 is an explanatory diagram showing an example of a menu screen displayed on the liquid crystal display 242 in step S102. FIG. 5A shows a menu screen MN1 for acquiring designation of a RAW image file, and FIG. 5B shows a menu screen MN2 for obtaining designation of a development parameter adjustment mode.

  As shown in FIG. 5A, the menu screen MN1 includes a folder selection area RFS for selecting a folder in which a RAW image file is stored, and six thumbnails TN1 to TN6. In the example of FIG. 5A, a folder named “Album / Photo / March 2008” is selected in the folder selection area RFS. Then, one thumbnail TN1 is selected from the thumbnails TN1 to TN6 of the RAW image file stored in the selected folder.

  When the user operates the OK button BOK (FIG. 3) in the state shown in FIG. 5A, the RAW image file corresponding to the thumbnail TN1 is designated as the RAW image file to be developed. When the cancel button BRT is operated in the state shown in FIG. 5A, the development processing is interrupted, and a menu screen (not shown) for selecting processing contents is displayed on the liquid crystal display 242.

  When the OK button BOK is operated in the state shown in FIG. 5A, a menu screen MN2 for acquiring designation of the adjustment mode shown in FIG. 5B is displayed. This menu screen MN2 shows two adjustment modes, “automatic adjustment mode” and “manual adjustment mode”. When the “automatic adjustment mode” is selected as shown in FIG. 5B, when the user operates the OK button BOK, the automatic adjustment mode is designated as the parameter adjustment mode. When the user presses the up / down direction button BDR (FIG. 3), the adjustment mode selection state is changed. When the user operates the cancel button BRT, the screen returns to the menu screen MN1 for acquiring the designation of the RAW image file.

  In step S104 of FIG. 4, the CPU 210 displays a reference display screen. Specifically, the CPU 210 reads screen nail image data from the RAW image file corresponding to the selected thumbnail TN1 (FIG. 5). Then, a reference display screen including an image (screen nail image) represented by the screen nail image data is displayed on the liquid crystal display 242.

  FIG. 6 is an explanatory diagram illustrating an example of a reference display screen. The reference display screen SAG includes a reference image display area RIG on which a screen nail image is temporarily displayed for reference, an adjustment mode setting area RMD, and a development parameter setting area RP0.

  As shown in FIG. 6, in the development parameter setting area RP0, “color temperature” and “color cast correction” representing white balance setting values, “exposure” representing an exposure correction coefficient, “hue” representing a hue correction amount, Slide bars are provided for “contrast” representing the contrast adjustment value, “sharpness” representing the sharpness adjustment value, and “noise removal” representing the degree of noise removal. In each of the slide bars, standard setting values of these parameters are indicated by triangular markers. As shown in FIG. 6, the slide bar is composed of a number line representing development parameters that can be expressed as a one-dimensional numerical value and a marker indicating the parameter value. From these development parameters, parameters used in actual development processing described later are generated. Therefore, it can be said that these parameters are parameters related to development parameters.

  The development parameter setting region RP0 of the reference display screen SAG is displayed in a light color (grayed out) in order to indicate that the entire development parameter setting region RP0 does not accept a change in the development parameter by the user. In this embodiment, even if the user performs an operation, the change is prohibited from being accepted by setting these markers not to move. The individual parameters displayed in the development parameter setting area RP0 will be described later.

A3. RAW image generation:
In step S106 of FIG. 4, the CPU 210 generates a RAW image. The generation of a RAW image is a process of taking out image data from the RAW data area in FIG. 2 and performing an expansion process on the compressed data. Here, the RAW image is an intermediate image before performing processes such as pixel interpolation and color conversion among various processes (described later) performed in development. The CPU 210 extracts RAW data from the RAW image file (FIG. 2), performs a predetermined process, and generates a RAW image.

  FIG. 7 is a flowchart showing an example of the RAW image generation process executed in step S106. Note that the RAW image generation processing shown in FIG. 7 is appropriately changed depending on the format of the RAW image data.

  In step S202, the CPU 210 analyzes the data format of the RAW image file (FIG. 2). As described above, the data format of the RAW image file differs depending on the manufacturer and model of the digital still camera 100 (FIG. 1). Therefore, the CPU 210 specifies the data format of the RAW image file from information such as the manufacturer and model number stored in the header section, and acquires the storage position and storage format of the RAW data. In the present embodiment, the data format of the RAW image file is specified based on the information stored in the header part of the RAW image file. However, depending on the extension attached to the RAW image file, the RAW image file may be RAW. It is also possible to specify the data format of the image file. For example, EPSN0012. If it is ERF, it may be a data format made by E company depending on the extension, or ADSC0125. If it is ARF, it is a method of specifying the data format made by Company A by the extension.

  In step S204, the CPU 210 extracts RAW data stored in the RAW image file and stores it in the RAM 230 (FIG. 3). Next, in step S206, data expansion processing is performed on the RAW data stored in the RAM. Usually, the RAW data is subjected to a reversible compression process (for example, Huffman coding) in order to reduce the data size. In step S206, processing (referred to as “decompression”) for converting the compressed data into data before compression is performed. When the Huffman coding compression of the previous example is performed, the Huffman expansion (decompression) process is performed based on the corresponding Huffman code data.

  In step S208, the CPU 210 performs inverse conversion (DPCM demodulation) of differential pulse code modulation (DPCM) performed when RAW data is generated. Next, in step S210, the compressed dynamic range is expanded.

  In step S212, the CPU 210 executes an optical black correction process. This process is a process of subtracting the offset value added to the RAW data from the RAW data in order to correct the characteristics of the imaging element of the camera, that is, the characteristic that the detected value does not become zero when the intensity of the incident light is zero. . In this process, the offset value is subtracted from the gradation value of each pixel included in the RAW data. For example, a value stored in the header part of the RAW image file (FIG. 2) can be used as the offset value to be subtracted. The output of the image sensor usually includes a noise component, and when this signal is A / D converted, it is usually A / D converted including the noise component. If the value corresponding to the RAW data when the intensity of the incident light is zero is set to zero, the noise in the positive direction is recorded as a positive value in the RAW data, and if it is a negative value, it is clamped to zero. This is a device for preventing the noise component from being accurately recorded and preventing the resulting noise component from having a positive value corresponding to the noise component. That is, when the value corresponding to the RAW data when the intensity of the incident light is zero is 64, for example, when the positive noise is +1, it is recorded as 65, and when the negative noise is −1, 63 To be recorded. Then, by subtracting the offset value 64, +1 and −1 are obtained, respectively. As a result, it is possible to correctly process RAW data including noise. If the RAW data is signed due to this negative noise, the bit recording width is added, and the problem that the amount of data increases can be solved. Thus, the data after optical black correction is calculated as a signed value having a negative value.

  In this way, a RAW image representing the amount of light incident on each pixel of the image sensor of the digital still camera 100 is generated from the RAW image file. In the following, the “RAW image” refers to data representing the amount of light incident on each pixel of the image sensor as described above. Note that a RAW image is usually represented by data in which the gradation value of each pixel is 12 bits or more. Accordingly, the arithmetic processing in the development processing is performed with 16 bits including the normal code so that information included in the RAW image is not lost. Therefore, in the following, it is assumed that arithmetic processing is performed with 16 bits unless otherwise specified. Of course, the bit length of the arithmetic processing is not limited to 16 bits, and may be another number of bits.

A4. Automatic adjustment of development parameters:
In step S108 in FIG. 4, the CPU 210 determines whether or not the development parameter adjustment mode is the automatic adjustment mode. If the parameter adjustment mode is the automatic adjustment mode, the process proceeds to step S110. On the other hand, when the parameter adjustment mode is the manual adjustment mode, the process proceeds to step S114. In this case, a predetermined standard parameter is set as the development parameter. Note that step S108 may be omitted, and step S110 may be executed after step S106 so that the development parameters are always adjusted automatically. In this case, the display of the menu screen MN2 in FIG. 5B in step S102 is omitted.

  In step S110, the CPU 210 performs development processing for analysis, and generates an analysis image for adjusting development parameters. FIG. 8 is a flowchart showing the flow of the development process for analysis executed in step S110.

  In step S302, the CPU 210 performs gain adjustment processing (pre-gain) on the RAW image. Specifically, the CPU 210 multiplies the pixel value (RGB value) of each pixel of the RAW image by a coefficient (pre-gain setting value) set according to the characteristics of the image sensor of the digital still camera 100. Since the gain difference between the RGB pixels is adjusted in the pre-gain, in general, the G pixel is not multiplied, and only the R and B pixels are multiplied by the pre-gain setting value. Note that the pre-gain setting value is normally stored in the header part of the RAW image file (FIG. 2). Therefore, in step S302, gain adjustment processing is performed using the pre-gain setting value stored in the header portion in this way. Note that the image on which gain adjustment processing has been performed (gain-adjusted image) is stored in the RAM 230 (FIG. 3), and gain adjustment processing is omitted by using the gain-adjusted image in the main development processing described later.

  In step S304, the CPU 210 performs a simple demosaic process. The demosaic processing is generally generated by interpolating color information that is missing from each pixel due to the arrangement of checkered color filters provided in the image sensor of the digital still camera 100 (FIG. 1). It is processing. However, in step S304, in order to generate an image having a resolution lower than that of the RAW image, a simple demosaic process for thinning out pixels is performed instead of the interpolation process. The image (reduced RGB image) obtained by performing the simple demosaic process on the RAW data is stored in the RAM 230 (FIG. 3), and the demosaic process is omitted in the display development process described later.

  FIG. 9 is an explanatory diagram showing a state in which simple demosaic processing is performed on a RAW image. As shown in FIG. 9A, the image sensor of the digital still camera 100 (FIG. 1) includes RGB color filters corresponding to the individual sensor elements formed on the light receiving surface of the image sensor. They are arranged in a pattern (referred to as “Bayer arrangement”). Therefore, the pixel of the RAW image corresponding to each sensor element of the image sensor is a pixel having color information of only one of RGB. For example, when a certain pixel is an R pixel, G and B color information at the position of the pixel is missing.

  In the simple demosaicing process, one pixel (RGB pixel) having all the RGB color information is generated from a 4 × 4 pixel area. Specifically, as shown in FIG. 9B, one R pixel, one G pixel, and one B pixel are selected from a 4 × 4 pixel region. Then, the RGB value of the selected pixel is set to the RGB value of the RGB pixel, and one RGB pixel is generated for the 4 × 4 pixel region as shown in FIG. 9C. As described above, by generating one RGB pixel from the 4 × 4 pixel region, a reduction process of ¼ can be performed in the simple demosaic process. It should be noted that the size of the pixel region used for generating one RGB pixel can be set as appropriate according to the reduction amount by the simple demosaic process. The size of the region for generating one RGB pixel can be set to 5 × 5, 3 × 3, or 2 × 2, for example. The R pixel, the G pixel, and the B pixel (used pixel) in the 4 × 4 pixel region used for generating the RGB pixel may be any pixel as long as it is a pixel in the 4 × 4 pixel region. It can be a pixel. In this embodiment, the pixel values of the R, G, and B selected pixels in the 4 × 4 pixel region are set to the RGB values of the RGB pixels. However, the RGB values of the RGB pixels are set by other methods. It is also possible to set. For example, an average value of pixel values of R, G, and B pixels in a 4 × 4 pixel region may be set as an RGB value of RGB pixels.

  In step S306 of FIG. 8, the CPU 210 adjusts the size of the reduced RGB image that has undergone the simple demosaic process. Specifically, the CPU 210 further trims and reduces the reduced RGB image reduced to ¼ from the RAW image by the simple demosaic process, and adjusts the size of the image to a size suitable for analysis.

  FIG. 10 is an explanatory diagram illustrating a state in which an image having a QVGA size (320 × 240 pixels) is generated from a RAW image having 3024 × 2016 pixels by the simple demosaic processing (step S306) and the image size adjustment processing (step S308). It is. In general, the ratio between the number of pixels in the horizontal direction and the number of pixels in the vertical direction of a RAW image of a digital still camera (referred to as “aspect ratio”) is set to 3: 2 in accordance with a 35 mm film camera. ing.

  The RAW image shown in FIG. 10A is reduced to ¼ by the simple demosaic process. Therefore, as shown in FIG. 10B, the size of the reduced RGB image is 756 × 504 pixels. Next, the reduced RGB image is subjected to trimming to cut off the left and right 42 pixels in order to adjust the aspect ratio to 4: 3 of the QVGA size. The size of the image after trimming indicated by the thick line in FIG. 10B is 672 × 504 pixels with an aspect ratio of 4: 3. The image whose aspect ratio is adjusted in this way is further subjected to reduction processing, and a QVGA size analysis RGB image shown in FIG. 10C is generated. As a reduction process for generating an analysis image from an analysis RGB image after trimming, for example, a known reduction process such as a nearest neighbor method or a bilinear method can be used.

  In step S308 of FIG. 8, the CPU 210 performs color reproduction processing on the analysis RGB image. Color reproduction processing is a process to correct the difference between the spectral sensitivity characteristics of the image sensor and the human visibility characteristics by adjusting the RGB values according to the human visibility characteristics, and to reproduce the correct colors. is there. The color reproduction process is performed by multiplying a three-dimensional vector having components of R value, G value, and B value by a 3 × 3 matrix having a correction coefficient as an element (3 × 3 matrix operation). This correction coefficient is usually different for each camera model. Therefore, as the correction coefficient, a correction coefficient prepared in advance corresponding to the camera model specified by the analysis of the RAW image file is used.

  In step S310, the CPU 210 performs tone correction (gamma correction) for correcting tone characteristics. In this embodiment, 2.2 used in normal gamma correction is used as the gamma value (γ) for performing gamma correction in the development processing for analysis. Note that the gradation correction in step S310 is substantially the same as the gradation correction performed in the main development process in step S122.

  In step S112 in FIG. 4, the CPU 210 analyzes an image (reduced image) generated by performing color reproduction processing and gamma correction on the analysis RGB image, that is, the image processed in step S110. Based on the analysis result of the reduced image, various development parameters such as an exposure correction coefficient, a white balance setting value, a hue correction value, and a contrast adjustment value are set.

  FIG. 11 is an explanatory diagram showing how the exposure correction coefficient is set by histogram analysis as an example of setting the development parameter in step S112. FIG. 11A and FIG. 11C show an image that is underexposed (underexposed) and an image that is appropriately exposed. FIG. 11B is a graph showing a histogram analysis result of the underexposed image shown in FIG. FIG. 11D is a graph showing a histogram analysis result of the image with the proper exposure shown in FIG. In the graphs shown in FIGS. 11B and 11D, the horizontal axis represents lightness (gradation value), and the vertical axis represents the number of pixels.

  As shown in FIG. 11B, in the underexposed image, the number of pixels in the high brightness area is reduced. In the example of FIG. 11B, most of the pixels are distributed in the low brightness area (0 to 136) in the entire brightness range (0 to 255), and the pixels in the high brightness area (137 to 255). The number is almost zero. On the other hand, as shown in FIG. 11D, in an image with proper exposure, the pixels are distributed over the entire range of brightness (0 to 255). Therefore, in order to change the underexposed image shown in FIG. 11 (a) to the properly exposed image shown in FIG. 11 (c), the shape of the histogram should cover the entire range of brightness as shown in FIG. 11 (d). In addition, it can be determined that exposure correction for converting the lightness 136 to the lightness 255 may be performed. The target histogram shape can be determined as a pixel distribution range.

As described above, the reduced image used for the analysis is obtained by performing gamma correction on the reduced RGB image. Specifically, the lightness Y of the reduced image is expressed by the following equation (1) using the lightness X of the reduced RGB image.
Y = X ^{1 / γ} (1)

Therefore, in order to change the underexposed image shown in FIG. 11 (a) to the appropriately exposed image shown in FIG. 11 (c), the correction coefficient A applied to the lightness of the RAW image is expressed by the following equation (2). Value to be set.
A = (255/136) ^γ (2)

  In gamma correction when generating a reduced image, γ is set to 2.2. Therefore, for the underexposed image shown in FIG. 11A, the correction coefficient A applied to the lightness of the RAW image is calculated to be about 4.0. In addition to the normally used 2.2 as the gamma value (γ), a contrast correction curve for picture creation and a brightness correction curve for absorbing the difference in brightness of each camera are provided. A synthesized gradation correction curve may be used. In this case, the curve used for tone correction is a composite curve obtained by synthesizing a contrast correction curve for picture creation or a brightness correction curve for absorbing the difference in brightness of each camera. For this reason, the correction coefficient A is obtained by performing inverse correction calculation of the composite curve, but the concept is the same as in equations (1) and (2).

  In this embodiment, the exposure correction coefficient is calculated from the result of the histogram analysis, but the exposure correction coefficient can also be obtained by other methods. For example, a face can be detected from the reduced image, and the exposure correction coefficient can be calculated based on the detected brightness of the face. In this case, the brightness of an area having a hue and saturation corresponding to the skin color around the nose, mouth and eyes of the detected face is evaluated. The exposure correction coefficient is determined so that the lightness of the evaluated area falls within a preferable brightness as the lightness of the skin color. The brightness of the evaluated area is calculated as described above so as to fall within the range of 180 to 200, for example.

  Other development parameters are also set by a method corresponding to each development parameter. For example, the white balance setting value is set so that the white area included in the reduced image is detected and the saturation of the detected white area is substantially zero. The hue correction value is set so that a specific type of subject (for example, sky or face) included in the reduced image is detected, and the color of the detected subject falls within the target color range. Also, the contrast adjustment value is set according to the type of subject specifying the subject represented by the reduced image. For example, when the subject is a landscape, the contrast adjustment value is set to a higher value, and when the subject is a person, the contrast adjustment value is set to a lower value.

A5. Display processing:
In step S <b> 114 of FIG. 4, the CPU 210 performs display development processing. FIG. 12 is a flowchart showing the flow of the development process for display executed in step S114. In the display development process, as described above, the reduced RGB image subjected to the simple demosaic process is processed.

  In step S402, the CPU 210 performs white balance adjustment processing. The white balance adjustment process is performed by multiplying the RGB value of each pixel constituting the reduced RGB image by a coefficient set for each target white balance. Specifically, the coefficients Ar and Ab are multiplied to the R value and B value of each pixel of the RAW image. These coefficients Ar and Ab are determined based on the white balance setting value set in step S112 in FIG. 4 or the white balance setting value (color temperature, color cast correction) changed by the user as will be described later. The These coefficients Ar and Ab can also be determined based on the white balance setting contents described in the header part in the RAW image file (FIG. 2).

  In step S404, the CPU 210 performs exposure correction processing. The exposure correction process is performed by multiplying the R value, the G value, and the B value of each pixel of the reduced RGB image by the same exposure correction coefficient Ga. Thus, multiplying each of the R value, G value, and B value by the same coefficient is equivalent to increasing or decreasing the amount of light incident on the image sensor of the digital still camera 100. Therefore, by performing the exposure correction process, it is possible to obtain the same effect as changing the exposure amount in the photographing stage of the digital still camera 100. As the exposure correction coefficient Ga, the exposure correction coefficient determined by the histogram analysis of the reduced image as described above, or the exposure correction coefficient changed by the user is used.

  In step S406, the CPU 210 performs color reproduction processing. The color reproduction process in step S406 is the same process as the color reproduction process in step S308 of the analysis development process shown in FIG. 8, and is performed by a 3 × 3 matrix operation.

  In step S408, the CPU 210 performs a hue correction process. Similar to the color reproduction process, the hue correction process is performed by a 3 × 3 matrix operation. Each element of the 3 × 3 matrix used for hue correction is set based on the target hue rotation angle. The hue rotation angle is determined based on the hue correction value set in step S112 in FIG. 4 or the hue correction value (hue) changed by the user as described later.

  Thus, in the white balance adjustment process and the exposure correction process, the RGB value is multiplied by a predetermined correction coefficient. In the color reproduction process and the hue correction process, a 3 × 3 matrix operation is performed. Since these processes are all linear operations, the arithmetic processing performed in steps S402 to S408 of the display development processing in FIG. 12 is expressed by the following equation (3).

  Here, the vector (r, g, b) represents the RGB value before the arithmetic processing performed in steps S402 to S408 (that is, the reduced RGB image), and the vector (r ′, g ′, b ′) is The RGB values after the arithmetic processing performed in steps S402 to S408 are represented. The matrix of Expression (3) represents a matrix used for white balance adjustment, exposure correction, color reproduction, and hue correction in order from the right. In this embodiment, each value of the vector (r, g, b) is a 12-bit numerical value (or a numerical value obtained by adding a sign to 12 bits), and each matrix element of the expression (3) is a signed 16-bit value. It is a numerical value.

  Therefore, in steps S402 to S408, a single matrix (correction processing matrix) is generated by multiplying these matrices in advance. Then, by performing the calculation processing of the following equation (4) using the correction processing matrix, each processing of white balance adjustment, exposure correction, color reproduction, and hue correction is performed collectively.

  Here, the matrix of Expression (4) is a correction processing matrix obtained by multiplying the matrix of Expression (3). Each element of the matrix of Equation (4) is also 16-bit signed accuracy, and the calculation of the element is performed with 32-bit accuracy so that saturation or accuracy does not decrease in the calculation process from Equation (3), Finally, it is summarized to 16-bit accuracy. The calculation of equation (4) is also performed so that saturation or the like does not occur.

  In step S410, the CPU 210 performs tone correction processing. In the tone correction process, the tone value of the reduced RGB image is corrected using a tone curve representing the relationship between the input tone value and the output tone value. This is determined based on the contrast adjustment value set in step S112 of FIG. 4 or the contrast adjustment value changed by the user as will be described later.

  In step S412, the CPU 210 performs gamma correction. As the gamma value (γ) used in the gamma correction in step S 412, a value corresponding to the display characteristics of the liquid crystal display 242 is used. In general, 2.2 is used as the gamma value (γ) corresponding to the display characteristics.

  In step S414, the CPU 210 performs color conversion processing. This color conversion process is a process of converting the RGB value of the RAW image, which is color information in the RGB space, into a color component value in the YUV color space that is accepted by the display controller 240 (FIG. 3). Similar to the color reproduction process (step S406, step S308 in FIG. 8) and the hue correction process (step S408), the color conversion process is also performed by a 3 × 3 matrix operation.

  In step S416, the CPU 210 performs noise removal processing. The noise removal process is a process for removing a noise component present in an image and generating a clear image. The noise removal process can be performed using, for example, a Gaussian filter.

  In step S418, the CPU 210 performs sharpness adjustment processing. Sharpness adjustment (that is, edge enhancement processing) is processing for correcting the blurring of the contour in the image due to the influence of the optical low-pass filter provided in the image sensor to clarify the image. Note that the noise removal process in step S416 and the sharpness adjustment process in step S418 may be performed before the tone correction in step S410. In this specification, “undeveloped image data” refers to data including data in the middle of a series of development processing on RAW data.

A6. Preview screen display:
In step S116 in FIG. 4, a preview screen including the image (display image) obtained by performing the display development process on the reduced RGB image is displayed on the liquid crystal display 242. After displaying the preview screen, the CPU 210 waits for a specific operation by the user. Specifically, when any of the change of the development parameter, the change of the development parameter adjustment mode to the automatic adjustment mode, and the instruction of the main development is performed, the process proceeds to step S118. Further, when the cancel button BRT (FIG. 3) is operated, the development processing of FIG. 4 is interrupted.

  In displaying the preview screen, the display image is appropriately trimmed and reduced. As shown in FIG. 10, when the size of the reduced RGB image is 756 × 504 pixels, the size of the display image is also 756 × 504 pixels. This display image is trimmed by cutting out two pixels at the top, bottom, left and right, and a display image of 752 × 500 pixels is displayed on the preview screen. Further, depending on the form of the liquid crystal display 242 (FIG. 3) or the preview screen, the trimmed display image is further reduced, and a smaller image (for example, an image of 600 × 400 pixels) is displayed on the preview screen. Is displayed.

  FIG. 13 is an explanatory diagram showing an example of the preview screen PV1 displayed on the liquid crystal display 242 in step S116. As shown in FIG. 13, the preview screen PV1 is provided with a display image display area RID in which a display image is displayed, an adjustment mode setting area RMD, and a development parameter setting area RP1.

  As shown in FIG. 13, in the development parameter setting area RP1, as in the reference display screen SAG (FIG. 6), “color temperature”, “color fog correction”, “exposure”, “tone”, “contrast”, Slide bars are provided for “sharpness” and “noise removal”, respectively. In the example of FIG. 13, among the automatically adjusted development parameters, “color temperature”, “color fog correction”, “exposure”, and “contrast” are adjusted to values different from the standard parameters. The marker is set at a position different from the standard parameter position. On the other hand, among the automatically adjusted development parameters, a marker is set at the position of the standard parameter for “hue” that has not changed from the standard parameter. Further, development parameters that are not automatically adjusted (sharpness and noise removal) remain standard parameters.

  As shown in FIG. 13, in the preview screen PV1, the grayout of the development parameter setting area RP1 is canceled to indicate that the change operation is accepted. When the change operation is accepted, the position of the marker of each slide bar is changed by the user's operation. The user can change the corresponding development parameter by operating the direction button BDR or the like provided on the photo viewer 200 (FIG. 3) to move the marker of each slide bar to the left or right.

  In step S118 in FIG. 4, the CPU 210 determines whether or not the operation by the user is to change the development parameter. If the user operation is an operation for changing the development parameter, the process returns to step S114, the display development process (step S114) is performed again based on the changed development parameter, and the redevelopment is performed in step S116. A preview screen including the displayed image is displayed.

  FIG. 14 is an explanatory diagram showing a preview screen that is displayed again after the development parameters are changed by the user. In the example of FIG. 14, the user changes the exposure correction coefficient to + 3EV, which is slightly higher than the set value (+ 2EV) by automatic adjustment, and the contrast is set to zero, which is slightly higher than the set value (−2.5) by automatic adjustment. The changed state is shown. As shown in FIG. 14, the marker on the slide bar in the development parameter setting area RP2 is moved according to the user's operation. The development parameter adjustment mode displayed in the adjustment mode setting area RMD is switched from the automatic adjustment mode to the manual adjustment mode in response to a parameter change by the user. The display image in the display image display area RID has been changed to an image that is brighter and higher in contrast than the image before the change.

  When the user changes the development parameter adjustment mode to the automatic adjustment mode, in step S118 of FIG. 4, the CPU 210 determines that the operation by the user does not change the development parameter. In step S120, CPU 210 determines that the operation by the user is a change to the automatic adjustment mode, and the process proceeds to step S126.

  In step S126, reception of an instruction to change the development parameter is prohibited. In addition, the development parameter setting area is grayed out to indicate that the change instruction is not accepted. FIG. 15 is an explanatory diagram showing a state in which the development parameter setting area of the preview screen PV2 is grayed out in step S126. The preview screen PV3 shown in FIG. 15 shows that the development parameter setting area RP3 is grayed out and that the adjustment mode shown in the adjustment mode setting area RMD is switched to the automatic adjustment mode. It is different from the screen PV2. The other points are the same as the preview screen PV2 shown in FIG.

  In step S126, after the acceptance of the development parameter change instruction is prohibited, the process returns to step S110, and the development of the analysis image and the development parameter setting are performed as described above. Then, the first preview screen PV1 is displayed on the liquid crystal display 242. In this embodiment, after step S126, the process is returned to step S110. However, the automatically adjusted development parameters are stored in advance in the RAM 230 (FIG. 3), and the display image is developed with the stored development parameters. (Step S114) may be performed. In this case, after step S126, the process returns to step S114.

  When the user operates the OK button in the state where the preview screen is displayed in step S116, the CPU 210 changes either the development parameter or the development parameter adjustment mode to the automatic adjustment mode in steps S118 and S120. But I judge it not. In step S122, the CPU 210 executes main development processing.

A7. Main development:
FIG. 16 is a flowchart showing the flow of the main development process executed in step S122. In the main development process, a process based on the designated development parameter is performed when the user operates the OK button. The main development process shown in FIG. 16 includes the point that step S401 is added before the white balance adjustment (step S402), the gamma value used in the gamma correction process (step S412), and the color conversion process (step S414). 12) is different from the display development processing shown in FIG. 12 in that the converted color space is set based on the format of the output data. Other points are the same as the display development processing shown in FIG. For this reason, in FIG. 16, illustrations after step S404 after white balance adjustment (step S402) are omitted.

  In step S401, the CPU 210 performs a demosaic process (predictive interpolation process) for main development on the gain-adjusted image generated in step S302 of FIG. 8 and stored in the RAM 230 (FIG. 3). The predictive interpolation process executed in step S401 is a demosaicing process that obtains the missing color information by predictive interpolation from the color information of surrounding pixels. In the predictive interpolation process, the direction of the edge of the image is detected, and pixel interpolation is performed along the edge of the image. For this reason, deterioration in image quality due to an interpolation error such as coloring of the periphery of the edge caused by interpolation using the color information of the pixel in the direction across the edge is suppressed.

  In step S124 in FIG. 4, the CPU 210 generates developed image data in a predetermined format from the image generated in the main development processing in step S122. The generated developed image data is stored in the HDD 270 or the memory card MC. When JPEG format image data is generated as developed image data, 16-bit data used in the calculation process of the main development process is converted to 8 bits when the developed image data is generated. The developed image data format does not necessarily have to be the JPEG format. In general, the developed image data can be in any format as long as it is a standard image data format such as TIFF format or BMP format. The developed image data may be stored as 16-bit TIFF data without being converted to 8-bit.

  Normally, even if the user changes the development parameter during the development parameter determination process, the change is changed to the parameter determined by the process when the development parameter determination process ends. . In this embodiment, during the period when the process for determining the development parameter by automatic adjustment or the like is being performed, the user is prohibited from accepting an instruction to change the development parameter, and the reference display screen, the preview screen, etc. The parameter setting area provided in the user interface is grayed out. For this reason, it is possible to suppress the useless operation of changing the development parameter in a state where the change is not reflected, so that the convenience of adjusting the development parameter is improved.

  In this embodiment, the entire parameter setting area is grayed out to indicate that the user is prohibited from accepting development parameter change instructions. However, only the markers provided on the slide bar or the slide bar are displayed. It may be grayed out. Generally, an area (change area) may be grayed out for changing development parameters provided on the user interface, such as the entire parameter setting area, a slide bar, or a marker. Further, instead of graying out the change area, the change area may be hidden to indicate that the user is prohibited from accepting a development parameter change instruction.

  Further, it is also possible to simply prohibit acceptance of a development parameter change instruction without graying out or displaying the change area. However, it is preferable that the change area is grayed out or not displayed because the user can grasp when the change parameter change instruction is not accepted.

B. Modified preview screen:
FIG. 17 is an explanatory diagram showing a first modification of the preview screen displayed on the liquid crystal display 242 in step S116. The preview screen PV4 shown in FIG. 17 differs from the preview screen PV2 of the embodiment in the display form of the parameter setting area RP4. Other points are the same as in the embodiment.

  As shown in FIG. 17, in the preview screen PV4 of the first modified example, a marker (reference marker) indicating a development parameter before being changed by the user is added above the slide bar provided in the parameter setting region RP4. ing. The added reference marker is set so as not to be changed by the user, and the user changes the development parameter by moving the marker below the slide bar. By adding the reference marker to the slide bar, the user can return the development parameter to the state before the change by moving the marker of the movable slide bar to the position of the reference marker. Thus, it becomes easier to return the development parameters to the state before the change, and the convenience of adjusting the development parameters is improved.

  FIG. 18 is an explanatory diagram showing a second modification of the preview screen displayed on the liquid crystal display 242 in step S116. The preview screen PV5 shown in FIG. 18 is different from the preview screen PV4 of the first modified example in that the color of the marker moved by the user is changed. Other points are the same as those of the first modification. Thus, by changing the color of the moved marker, the user can intuitively grasp the changed development parameter, and the convenience of adjusting the development parameter is improved. Note that the color of the marker may be changed according to the difference in development parameters before and after the change. In this way, the user can grasp the change amount of the development parameter based on the marker color, and the convenience of the development parameter adjustment is further improved.

  FIG. 19 is an explanatory diagram showing a third modification of the preview screen displayed on the liquid crystal display 242 in step S116. A preview screen PV6 shown in FIG. 19 is provided with a text box set to be non-rewritable representing parameters before the change by the user and an edit box for inputting development parameters. This is different from the preview screen PV4. Other points are the same as those of the first modification. In this way, by providing a text box that represents the parameter before the change and an edit box for inputting the development parameter, it is possible to directly input a number or the like as the development parameter, so that the development parameter is set to a desired value. It becomes easier to change. Further, by displaying the development parameter before the change in the text box, the development parameter can be returned to the state before the change. Thus, it becomes easier to return the development parameters to the state before the change, and the convenience of adjusting the development parameters is improved.

  FIG. 20 is an explanatory diagram showing a fourth modification of the preview screen displayed on the liquid crystal display 242 in step S116. On the preview screen PV7 shown in FIG. 20, the slide bar is arranged so that the development parameters set by image analysis or the like are positioned on the reference line indicated by the dotted line. Thereby, the user can return the development parameter to the state before the change by returning the marker to the reference line. Thus, it becomes easier to return the development parameters to the state before the change, and the convenience of adjusting the development parameters is improved.

  FIG. 21 is an explanatory diagram showing a fifth modification of the preview screen displayed on the liquid crystal display 242 in step S116. In the parameter setting area RP8 of the preview screen PV8 shown in FIG. 21, “color temperature”, “color fog correction”, “exposure”, and “tone” are replaced with a slide bar for changing absolute values. A slide bar for fine adjustment of each is provided. Other points are the same as those of the fourth modification shown in FIG. As shown in FIG. 21, in the fifth modified example, the development parameter set by image analysis or the like is located on the reference line indicated by the dotted line, so it is easier to return the development parameter to the state before the change. . Further, by making fine adjustments for development parameters that are greatly affected by changes to the image instead of changing the absolute values, the user can set appropriate development parameters more easily. .

C. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

C1. Modification 1:
In each of the above embodiments, the present invention is applied to the development processing in the photo viewer. However, the present invention can be applied to any apparatus in which the development processing is performed. The present invention can be applied to, for example, a personal computer or a printing apparatus that prints an image as long as the apparatus has a development processing function.

1 is a schematic configuration diagram schematically showing a configuration of an image display system to which an embodiment of the present invention is applied. Explanatory drawing which shows the data format of the RAW image file produced | generated by a digital still camera. The schematic block diagram which shows the structure of a photo viewer roughly. 6 is a flowchart showing a flow of development processing. Explanatory drawing which shows an example of the menu screen displayed on a liquid crystal display in step S102. Explanatory drawing which shows an example of a reference display screen. The flowchart which shows an example of the RAW image generation process performed in step S106. 7 is a flowchart showing the flow of analysis development processing executed in step S110. Explanatory drawing which shows a mode that a simple demosaic process is performed to a RAW image. Explanatory drawing which shows a mode that the image of QVGA size is produced | generated from a RAW image. Explanatory drawing which shows a mode that the exposure coefficient among development parameters is set. The flowchart which shows the flow of the display development process performed in step S114. Explanatory drawing which shows an example of the preview screen PV1 displayed on a liquid crystal display in step S116. Explanatory drawing which shows the preview screen displayed again, after the development parameter is changed by the user. Explanatory drawing which shows the state by which the development parameter setting area | region of the preview screen was grayed out in step S126. 6 is a flowchart showing the flow of the main development process. Explanatory drawing which shows the 1st modification of a preview screen. Explanatory drawing which shows the 2nd modification of a preview screen. Explanatory drawing which shows the 3rd modification of a preview screen. Explanatory drawing which shows the 4th modification of a preview screen. Explanatory drawing which shows the 5th modification of a preview screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image data generation system 100 ... Digital still camera 200 ... Photo viewer 210 ... CPU
220 ... ROM
230 ... RAM
240: Display controller 242, Liquid crystal display 250: Input port 252, Operation button 260: Memory card interface 262: Memory card slot 270: HDD
BDR ... Direction button BRT ... Cancel button BZD ... Reduce button BZU ... Enlarge button MC ... Memory card

Claims (7)

A development processing device for developing undeveloped image data generated by a digital camera,
A development processing unit that generates developed image data in a predetermined format from the undeveloped image data by performing development processing on the undeveloped image data;
A development parameter determination unit for determining a first parameter related to the development parameter used in the development processing;
A change screen display for displaying a parameter change screen;
With
The change screen display unit displays the first parameter determined by the development parameter determination unit and the second parameter changed by the user on the parameter change screen,
The development processing unit performs development processing based on at least the second parameter. The development processing apparatus according to claim 1,
The first parameter can be expressed as a one-dimensional numerical value;
The change screen display section
Displaying the first parameter as a first marker image indicating a position on a number line corresponding to the first parameter displayed on the parameter change screen, which is set to be unmovable by the user;
A development processing apparatus that displays the second parameter as a second marker image that indicates a position on the number line that is set to be movable by a user. The development processing apparatus according to claim 2,
The second marker image is displayed in different colors when the first parameter and the second parameter are the same, and when the first parameter and the second parameter are different. Development processing device. The development processing apparatus according to claim 3,
The color of the second marker image changes according to the difference between the first parameter and the second parameter. The development processing apparatus according to claim 1, wherein:
The change screen display section
The first parameter is displayed on the parameter change screen as a character string that cannot be rewritten by the user, and the second parameter is displayed on the parameter change screen as a character string that can be rewritten by the user. Development processing device. A development processing method for developing undeveloped image data generated by a digital camera,
By developing the undeveloped image data, a developed image data of a predetermined format is generated from the undeveloped image data,
Determining a first parameter relating to a development parameter used in the development processing by a predetermined method;
Display the parameter change screen,
Displaying the first parameter determined by the predetermined method and the second parameter changed by the user on the parameter change screen;
The development process is performed based on at least the second parameter. A computer program for developing processing for developing undeveloped image data generated by a digital camera,
A function of generating developed image data in a predetermined format from the undeveloped image data by performing development processing on the undeveloped image data;
A function of determining a first parameter related to a development parameter used for the development processing by a predetermined method;
A function to display a parameter change screen;
Causing the computer to realize a function of displaying the first parameter determined by the predetermined method and the second parameter changed by the user on the parameter change screen;
The development program is a computer program that is performed based on at least the second parameter.

Images