JP2008283286A - Development processor for undeveloped image data, development processing method, and computer program for development processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it easier to set development conditions when primarily developed image data are generated by developing undeveloped image data. <P>SOLUTION: A development processor performs development processing on an image file including the undeveloped image data and attached image data attached thereto. The development processor includes a primary development processing unit which generates primarily developed image data from the undeveloped image data, a first development processing unit for analysis which generates first image data for analysis of predetermined resolution, and a second development processing unit for analysis which generates second image data for analysis of predetermined resolution from the attached image data. An image data selector for analysis selects image data for analysis based upon priority levels of the first and second image data for analysis. A development condition determining unit analyzes the selected image data for analysis to determine development conditions to be used by the primary development processing unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for developing undeveloped image data generated by a digital camera.

  In recent years, in a digital still camera, RAW image data (undeveloped image data) in which data representing the amount of light incident on an image sensor such as a CCD provided in the digital still camera is recorded as it is is recorded on a memory card. The RAW image data recorded on the memory card in this way is image data such as JPEG format and TIFF format that are normally used, image data for printing, etc. (mainly developed image data) by a development processing application of a personal computer. Is converted to

JP-A-8-298669 JP 2006-295815 A JP 2006-227698 A

  However, in the process of converting RAW image data into image data in a format that is normally used (referred to as “development process”), the generated developed image data generated by the process parameters (development conditions) during the development process The image quality changes greatly. Therefore, there has been a problem in that preferable developed image data cannot always be obtained unless appropriate development conditions are set. Further, even when good development conditions are found for specific RAW image data, the conditions at the time of shooting a RAW image are different, so that the development conditions are not necessarily preferred development conditions for other images. was there.

  The present invention has been made to solve the above-described problems in the prior art, and is a technique that makes it easier to set development conditions when developing undeveloped image data to generate main developed image data. The purpose is to provide.

  In order to solve at least a part of the above problems, a development processing apparatus of the present invention provides undeveloped image data generated by a digital camera and associated image data which is developed image data associated with the undeveloped image data. A development processing apparatus that executes development processing based on an image file including: a main development processing unit that generates main development image data in a predetermined format from the non-development image data; By performing development processing, a first analysis image generation unit that generates first analysis image data having a predetermined resolution and second analysis image data having a predetermined resolution are generated from the accompanying image data. In the main development processing unit, by analyzing any one of the second analysis image generating unit, the first analysis image data, and the second analysis image data A developing condition determining unit that determines a developing condition to be used; and an analysis by the developing condition determining unit based on a priority order between the first analysis image data and the second analysis image data. And an analysis image data selection unit for selecting analysis image data to be selected from either the first analysis image data or the second analysis image data.

  According to this configuration, the development conditions for generating the developed image data are set by analyzing either the first analysis image data or the second analysis image data. Therefore, it becomes easier to set the development conditions used in the development processing by the development processing unit. Further, when the accompanying image data accompanying the undeveloped image data is suitable for analysis, the second processing image data is generated from the accompanying image data, thereby calculating the analysis image data. The amount can be reduced. On the other hand, if the accompanying image data accompanying the undeveloped image data is not suitable for the analysis, the first analysis image data is generated from the undeveloped image data, so that the analysis for setting the development conditions can be further performed. It can be done reliably.

  The analysis image data selection unit may determine the priority order based on information about the digital camera included in the image file.

  Usually, the state of the accompanying image data accompanying the undeveloped image data differs depending on the digital camera. Therefore, by determining the priority order based on information about the digital camera, the selection of the image data for analysis becomes easier.

  The analysis image data selection unit may determine the priority order based on a type of the accompanying image data.

  When the accompanying image data is data representing a black and white image, the analysis image selection unit sets the priority of the first analysis image data higher than the priority of the second analysis image data. It is good also as what to do.

  Since black and white images lack color information, they are not suitable for determining development conditions depending on the development condition determination method by the development condition determination unit. Therefore, the development conditions can be determined more reliably by making the priority of the first analysis image data higher than the priority of the second analysis image data.

  According to this configuration, the priority order is determined based on the format of the accompanying image data accompanying the undeveloped image data. Therefore, since the state of the accompanying image data accompanying the undeveloped image data is more accurately reflected in the selection of the analysis image data, the analysis image data can be selected more accurately.

  The development condition determination unit includes a selection specification acquisition unit that acquires specification of selection of the image data for analysis by a user, and determines the priority based on the specification by the user acquired by the selection specification acquisition unit It is good.

  According to this configuration, the priority order is selected by the user's designation. Therefore, it is possible to make the developed image data more suitable for the user's preference.

  The resolution of the first analysis image data and the resolution of the second analysis image data may be the same.

  In the case of analyzing either the first analysis image data or the second analysis image data by making the resolution of the first analysis image data and the resolution of the second analysis image data the same. In addition, the development condition determination unit can determine the development conditions by the same processing. Therefore, the configuration of the development condition determining unit can be further simplified.

  The present invention can be realized in various modes. For example, the development processing apparatus and the development processing method, the image output apparatus and the image output method using the development processing apparatus or the development processing method, and the like. The present invention can be realized in the form of a computer program for realizing the function of the method or apparatus, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Third embodiment:
D. Fourth embodiment:
E. Example 5:
F. Example 6:
G. Variation:

A. First embodiment:
FIG. 1 is a schematic configuration diagram schematically showing the configuration of an image processing system 10 to which one embodiment of the present invention is applied. The image processing system 10 includes a digital still camera 100, a personal computer 200, and a printer PRT. The digital still camera 100 includes a memory card slot that stores a memory card MC indicated by an alternate long and short dash line.

  The digital still camera 100 generates an image file in a predetermined format from the captured image. The generated image file is stored in the memory card MC. The image file stored in the memory card MC is read by the personal computer 200 by inserting the memory card MC into the personal computer 200. The image data stored in the read image file is subjected to image processing by the personal computer 200 and printed by a printer PRT connected to the personal computer 200 with a cable CV. Alternatively, the memory card MC may be inserted into the printer PRT, and the printer PRT may execute printing based on the image file. It is also possible to transfer an image file from the digital still camera 100 to the personal computer 200 by connecting the personal computer 200 and the digital still camera 100 with a cable CV.

  FIG. 2 is an explanatory diagram showing a data format of a RAW image file generated by the digital still camera 100. Here, the RAW image file is a kind of image file generated by the digital still camera 100 and stores RAW image data representing an output value of an image sensor (CCD, CMOS, etc.) included in the digital still camera 100. A file. This RAW image file is created in a data format similar to the Exif (Exchangeable Image File Format) format, and includes a header portion and a data portion as shown in the figure.

  In the header section, additional information such as the manufacturer name and model number of the camera that created the RAW image file, shooting conditions, shooting date and time, and device characteristics are described. Note that the shooting conditions include setting conditions at the time of shooting such as shutter speed, aperture value, white balance setting value, and scene setting. Further, the device characteristics include various parameters representing the characteristics of the device of the digital still camera 100 such as an offset value (described later) with respect to the gradation value.

  The data portion includes RAW image data and display image data generated at the time of shooting. Display image data is image data (developed image data) obtained by performing development processing (image generation processing) on RAW image data. Note that the same subject (captured image) is represented in the RAW image data and the display image data, and the RAW image data and the display image data are recorded at the same time as shooting. The display image data is, for example, JPEG format image data that is used when a captured image is simply displayed on a display panel provided in the camera and has undergone development processing in the camera, and is the same as a RAW image. It may be an image having vertical and horizontal pixels, or may be a pixel that has been reduced vertically and horizontally for display. The display image data may be reduced to a size of about VGA (640 × 480 pixels). This display image data is also called screen nail image data. Since this screen nail image data is image data associated with RAW image data, it can also be referred to as “accompanying image data”. The data format of the RAW image file differs depending on the manufacturer and model of the digital still camera 100 (FIG. 1).

  FIG. 3 is a schematic configuration diagram schematically showing the configuration of the personal computer 200. The personal computer 200 includes a central processing unit (CPU) 210, a display 220 for displaying images, a peripheral device interface 230, a memory card interface 240, and an internal storage device 250. The CPU 210, the display 220, the peripheral device interface 230, and the internal storage device 250 are connected to each other via the bus 202, and data exchange between these units 210 to 230, 250 is performed via the bus 202. Done. The memory card interface 240 is connected to the peripheral device interface 230, and exchange of data between the memory card MC inserted into the memory card interface 240 and the personal computer 200 is performed via the peripheral device interface 230. .

  The internal storage device 250 stores a development processing unit 300. The development processing unit 300 includes a RAW image acquisition unit 310, a screen nail reduction unit 320, a reduced image analysis unit 330, a display development processing unit 340, and a main development processing unit 350. These units 310 to 350 are stored in the internal storage device 250 as program modules. The function as the development processing unit 300 is realized by the CPU 210 executing the program modules 310 to 350 in the development processing unit 300. The development processing unit 300 performs batch development processing for performing development processing on a plurality of RAW image data stored in each of a plurality of RAW image files, and development on RAW image data stored in one RAW image file. It has two development processing functions, that is, individual development processing for performing processing. The internal storage device 250 also includes an intermediate data storage unit 410 that temporarily stores data generated during the execution of development processing, and a developed image storage unit 420 that stores image data subjected to development processing. , Is provided.

  The peripheral device interface 230 is connected to an input device such as a keyboard and a mouse (not shown), a digital still camera 100 (FIG. 1), a printer PRT (FIG. 1), and the like. The user can give an instruction to the personal computer 200 by operating an input device connected to the peripheral device interface 230.

  FIG. 4 is a flowchart showing a flow of batch development processing for performing development processing on a plurality of RAW image files under development conditions set in a batch. This batch development process is executed by the development processing unit 300. The batch development processing and the individual development processing described later are executed when the user starts a development application for performing development processing on a RAW image file or when the development processing is instructed in the image processing application.

  In step S110, the development processing unit 300 (FIG. 3) acquires an instruction (development designation) for designating development contents. As the development contents, a RAW image file storing RAW image data to be developed and development conditions are designated. Specifically, the development processing unit 300 displays on the display 220 (FIG. 3) a menu screen (hereinafter referred to as “development content designation screen”) that prompts the user to designate a development target. A user operates the pointing device such as a mouse while viewing the menu screen displayed on the display 220 to instruct the development contents to the personal computer 200.

  FIG. 5 is an explanatory diagram showing an example of the development content designation screen SDC displayed in step S110. The development content designation screen SDC is provided with a folder selection area RFS for selecting a folder in which a RAW image file is stored and an image selection area RIS for selecting a RAW image file in the selected folder. It has been. In the example of FIG. 5, a folder named “November 27, 2006” is selected in the folder selection area RFS, and two RAW image files are selected in the image selection area RIS. When the user selects the development target in this way and operates the “development start” button BDS, the RAW image data stored in each of the selected RAW development files is subjected to development processing, and developed image data is generated. Is done. When the user operates the “return” button BRT, the batch development process is interrupted.

  In addition to these selection areas RFS and RIS, process designation areas RMI, RBI, and REI for designating development conditions are provided on the development content designation screen SDC. The user can designate development conditions by operating a pointing device in these process designation areas RMI, RBI, and REI. In the example of FIG. 5, “collective development” is designated in the mode designation region RMI that designates the operation mode (development mode) of the development process. When the user operates the “individual development” mark, an individual development process described later is executed.

  In the basic setting area RBI, development conditions (basic settings) that are normally used in the development processing are designated. In the example of FIG. 5, “automatic exposure correction” that automatically sets an exposure correction parameter when performing development processing among the basic settings is ON. The style for setting standard parameters other than the exposure correction of the development process is set to “landscape”. In the extended setting area REI, more detailed development conditions (extended settings) are specified in addition to the basic settings. As extended settings, five items of “exposure”, “color temperature”, “hue”, “contrast” and “sharpness” can be set. These setting contents will be described later. In the example of FIG. 5, since the basic setting style is set to “landscape”, “sharpness” is set two steps lower in the setting items of the extended setting.

  FIG. 6 is an explanatory diagram showing an example different from FIG. 5 of the development content designation screen SDC displayed in step S110. In the development content designation screen SDC of FIG. 6, “automatic exposure correction” is set to OFF among the basic setting items, and “exposure” is set one step higher among the setting items of the extended setting. This is different from the development content designation screen SDC shown in FIG. When the user operates the “development start” button BDS on the development content designation screen SDC in FIG. 6, the RAW image data stored in each of the selected RAW development files according to the development conditions designated by each item of the extension setting. Is subjected to development processing, and developed image data is generated.

  In step S120 of FIG. 4, the RAW image acquisition unit 310 acquires RAW image data from one of the RAW image files specified in step S110. FIG. 7 is an example of a flowchart showing the flow of the RAW image acquisition process executed in step S120. (The processing flow in FIG. 7 varies depending on the RAW image file format and the like.)

  In step S210, the RAW image acquisition unit 310 analyzes the data format of the RAW image file (FIG. 2). As described above, the data format of the RAW image file differs depending on the manufacturer and model of the digital still camera 100 (FIG. 1). Therefore, the RAW image acquisition unit 310 specifies the data format of the RAW image file from information such as the manufacturer and model number stored in the header portion, and acquires the storage position and storage format of the RAW image data. In the present embodiment, the data format of the RAW image file is specified based on the information stored in the header part of the RAW image file. However, depending on the extension attached to the RAW image file, the RAW image file may be RAW. It is also possible to specify the data format of the image file. For example, EPSN0012. If it is ERF, it may be a data format made by E company depending on the extension, or ADSC0125. If it is ARF, it is a method of specifying the data format made by Company A by the extension.

  In step S220, the RAW image acquisition unit 310 extracts the RAW image data stored in the RAW image file and stores it in the intermediate data storage unit 410 (FIG. 3). Next, in step S230, the RAW image data stored in the intermediate data storage unit 410 is subjected to data expansion processing. Usually, RAW image data is subjected to a reversible compression process (for example, Huffman coding) in order to reduce the data size. In step S230, processing (referred to as “decompression”) for converting the compressed data into data before compression is performed. When the Huffman coding compression of the previous example is performed, the Huffman expansion (decompression) process is performed based on the corresponding Huffman code data.

  In step S240, the RAW image acquisition unit 310 performs inverse conversion (DPCM demodulation) of differential pulse code modulation (DPCM) that is performed when RAW image data is generated. Next, in step S250, the compressed dynamic range is expanded.

  In step S260, the RAW image acquisition unit 310 performs an optical black correction process. This process is a process of subtracting the offset value added to the RAW data from the RAW data in order to correct the characteristics of the imaging element of the camera, that is, the characteristic that the detected value does not become zero when the intensity of incident light is zero. . In this process, the offset value is subtracted from the gradation value of each pixel included in the RAW image data. For example, a value stored in the header part of the RAW image file (FIG. 2) can be used as the offset value to be subtracted. The output of the image sensor usually includes a noise component, and when this signal is A / D converted, it is usually A / D converted including the noise component. If the value corresponding to the RAW data when the intensity of the incident light is zero is set to zero, the noise in the positive direction is recorded as a positive value in the RAW data, and if it is a negative value, it is clamped to zero. This is a device for preventing the noise component from being accurately recorded and preventing the resulting noise component from having a positive value corresponding to the noise component. That is, when the value corresponding to the RAW data when the intensity of the incident light is zero is 64, for example, when the positive noise is +1, it is recorded as 65, and when the negative noise is −1, 63 To be recorded. Then, by subtracting the offset value 64, +1 and −1 are obtained, respectively. As a result, it is possible to correctly process RAW data including noise. If the RAW data is signed due to this negative noise, the bit recording width is added, and the problem that the amount of data increases can be solved. Thus, the data after optical black correction is calculated as a signed value having a negative value.

  In this way, RAW image data representing the amount of light incident on each pixel of the image sensor of the digital still camera 100 is acquired from the RAW image file. In the following, “RAW image data” refers to RAW image data that represents the amount of light incident on each pixel of the image sensor as described above. In addition, since the RAW image data represents an image of a subject formed on the image sensor, the RAW image data is also simply referred to as “RAW image” below. Note that the RAW image data is usually represented by data having a gradation value of 12 bits or more for each pixel. Accordingly, the arithmetic processing in the development processing is performed with 16 bits including the normal code so that information included in the RAW image data is not lost. Therefore, in the following, it is assumed that arithmetic processing is performed with 16 bits unless otherwise specified. Of course, the bit length of the arithmetic processing is not limited to 16 bits, and may be another number of bits.

  In step S130 of FIG. 4, the development processing unit 300 (FIG. 3) determines whether or not automatic exposure correction is set. Specifically, it is determined whether or not the automatic exposure correction is set by referring to the development designation acquired in step S110. If it is determined that automatic exposure correction is set, control is transferred to step S140. On the other hand, if it is determined that automatic exposure compensation is not set, control is transferred to step S160. In the example of FIG. 5, since the item “automatic exposure correction” is ON, it is determined in step S130 that automatic exposure correction is set, and step S140 is executed. On the other hand, in the example of FIG. 6, since the item of “automatic exposure correction” is OFF, it is determined in step S130 that automatic exposure correction is not set, and step S160 is executed.

  In step S140, the screen nail reduction unit 320 generates a scaled image having a predetermined size (resolution) by scaling the screen nail image represented by the screen nail image data (FIG. 2). The generated scaled image is used in image analysis described later. In the present embodiment, a QVGA size image having 320 × 240 pixels is generated as a scaled image. Regardless of the resolution of the screen nail image, image analysis is facilitated by setting the scaled image to a predetermined resolution. Note that the resolution of the scaled image may be other than 320 × 240 pixels. The resolution of the scaled image may be a size suitable for image analysis. The resolution of the zoomed image is preferably a smaller image as long as it has a size suitable for image analysis in order to reduce the amount of calculation processing required for image analysis. If there is a margin in processing speed, processing may be performed with the screen nail size maintained without performing scaling processing. In this case, the process of step S150 is directly executed without performing the process of step S140.

  FIG. 8 is an explanatory diagram showing a state in which a zoomed image is generated from the screen nail image. In the example of FIG. 8, the screen nail image is an image of 3008 × 2000 pixels. In this embodiment, a 320 × 240 pixel scaled image is generated. Usually, since the screen nail image is larger than the zoomed image (having a higher resolution), the reduction process is performed. Therefore, hereinafter, the scaled image is also referred to as a reduced image. From the screen nail image shown in FIG. 8A, a region corresponding to 172 pixels at the left and right ends and a region corresponding to one pixel at both the upper and lower ends are cut off in accordance with the 4: 3 aspect ratio of the reduced image. An image of 2664 × 1998 pixels shown in 8 (b) is generated. Next, the 2664 × 1998 pixel image is reduced to 1 / 8.325, and a 320 × 240 pixel reduced image shown in FIG. 8C is generated. The reduction of the image is performed using, for example, a bilinear method. However, it is also possible to use a simple thinning method that reduces an image by thinning out pixels. When reducing the image, it is more preferable to apply a low-pass filter to reduce the high-frequency component of the spatial frequency in order to suppress the occurrence of aliasing noise.

  When the screen nail image data is JPEG format data, instead of generating a pixel block of 8 × 8 pixels by decompressing the JPEG format data, a block having a smaller number of pixels (for example, 5 × 5 pixels, 4 × 4) It is also possible to reduce the screen nail image by generating a 3 × 3 pixel block).

  When the number of pixels of the screen nail image is different, the area to be cut off and the reduction ratio are changed as appropriate, and a reduced image of 320 × 240 pixels is generated. For example, when the screen nail image is 1504 × 1000 pixels, the left and right 86 pixel areas and the lower one pixel area are cut off and reduced to 1 / 4.1625. Further, when the screen nail image is 640 × 424 pixels, the region for each of the left and right 38 pixels and the region for the lower one pixel are cut off and reduced to 1 / 1.625. At this time, since the generation of the reduced image is intended to analyze the image, it is desirable to reduce the entire image. For this reason, it is preferable that the top, bottom, left, and right are cut off evenly so that the center of the original image remains as a result of cutting off, and a large range remains. When a 3: 2 aspect image is cut out at a ratio of 4: 3, the image is cut out equally on the left and right.

  In step S150 of FIG. 4, the reduced image analysis unit 330 analyzes the reduced image generated in step S140. Then, development conditions are set based on the analysis result of the reduced image. Specifically, histogram analysis of the reduced image is performed, and an exposure correction coefficient to be multiplied by each RGB gradation value (RGB value) of the RAW image is calculated based on the result of the histogram analysis.

  FIG. 9 is an explanatory diagram showing the state of histogram analysis in step S140. FIGS. 9A and 9C show an image with underexposure (underexposure) and an image with proper exposure, respectively. FIG. 9B is a graph showing a histogram analysis result of the underexposed image shown in FIG. FIG. 9D is a graph showing a histogram analysis result of the image with the proper exposure shown in FIG. In FIG. 9B and FIG. 9D, the horizontal axis represents lightness (tone value), and the vertical axis represents the number of pixels.

  As shown in FIG. 9B, in the underexposed image, the number of pixels in the high brightness area is reduced. In the example of FIG. 9B, most of the pixels are distributed in the low brightness area (0 to 136) in the entire brightness range (0 to 255), and the pixels in the high brightness area (137 to 255). The number is almost zero. On the other hand, as shown in FIG. 9D, in an image with appropriate exposure, the pixels are distributed over the entire range of brightness (0 to 255). Therefore, in order to change the underexposed image shown in FIG. 9 (a) to the properly exposed image shown in FIG. 9 (c), the shape of the histogram should cover the entire range of brightness as shown in FIG. 9 (d). In addition, it can be determined that exposure correction for converting the lightness 136 to the lightness 255 may be performed. The target histogram shape can be determined as a pixel distribution range.

Normally, a screen nail image is a RAW image that has been subjected to gamma correction. Specifically, the brightness Y of the screen nail image is expressed by the following equation (1) using the brightness X of the RAW image.
Y = X ^{1 / γ} (1)

Therefore, in order to change the underexposed image shown in FIG. 9A to an appropriately exposed image shown in FIG. 9C, the correction coefficient A applied to the lightness of the RAW image is expressed by the following equation (2). Value to be set.
A = (255/136) ^γ (2)

  In normal gamma correction, γ is set to 2.2. For this reason, the correction coefficient A applied to the brightness of the RAW image shown in FIG. 9A is calculated to be about 4.0.

  In this embodiment, the exposure correction coefficient is calculated from the result of the histogram analysis, but the exposure correction coefficient can also be obtained by other methods. For example, when the reduced image analysis unit 330 has a function of detecting a face, the exposure correction coefficient can be calculated based on the brightness of the face detected from the reduced image. Specifically, the brightness of an area having a hue and saturation corresponding to the skin color around the nose, mouth, and eyes of the detected face is evaluated. The exposure correction coefficient is calculated as described above so that the lightness of the evaluated region falls within the range of 180 to 200 that is preferable as the lightness of the skin color. Note that adjusting the lightness to fall within the range of 180 to 200 corresponds to adjusting the skin color so that the hue range is an ideal skin color and the lightness is within the color range of 180 to 200. Of course, the hue and saturation may also be adjusted.

  In step S160 of FIG. 4, the main development processing unit 350 executes the development process based on the development conditions set in step S110 and the development conditions set in step S150. FIG. 10 is a flowchart showing the flow of the main development process executed in step S160.

  In step S302, the main development processing unit 350 performs gain adjustment (pre-gain). Specifically, the development processing unit 350 multiplies the pixel value (RGB value) of each pixel constituting the RAW image by a coefficient (pre-gain setting value) set according to the characteristics of the image sensor of the digital still camera 100. To do. As the pre-gain setting value, a value stored in the header part of the RAW image file (FIG. 2) is used. In many cases, the pre-cane setting value is multiplied only for the R and G pixels.

  In step S304, the main development processing unit 350 performs pixel interpolation processing (demosaic processing) for printing. This pixel interpolation process is a process for interpolating and generating missing color component information due to the arrangement of a plurality of sensor elements provided in the image sensor of the digital still camera 100 (FIG. 1). In general, in an image sensor to which RGB primary color filters are attached, the color filters on the sensor elements are arranged in a checkered pattern (referred to as a “Bayer array”). Therefore, the pixel of the RAW image corresponding to each sensor element of the image sensor is a pixel having color information of only one of RGB. For example, when a certain pixel is an R pixel, G and B color information at the same position is insufficient. In the pixel interpolation process, this missing color information is obtained by predictive interpolation from the color information of surrounding pixels. In predictive interpolation, the edge direction of an image is detected, and pixel interpolation is performed along the edge of the image, so that deterioration in image quality due to errors or coloring during interpolation is suppressed.

  In step S306, the main development processing unit 350 performs white balance adjustment. The white balance adjustment is set for each target white balance with RGB gradation values (hereinafter, each gradation value is also referred to as R value, G value, B value) of each pixel constituting the RAW image. This is done by multiplying the coefficient. Specifically, the coefficients Ar and Ab are multiplied to the R value and B value of each pixel of the RAW image. These coefficients Ar and Ab are the contents of the “color temperature” item set in the white balance setting contents described in the header section in the RAW image file or the development contents designation screen SDC (FIGS. 5 and 6). Determined based on the setting contents. It is also possible to calculate these coefficients Ar and Ab from the result of image analysis.

  In step S308, the main development processing unit 350 performs exposure correction. Exposure correction is performed by multiplying the R value, G value, and B value of each pixel of the RAW image by the same exposure correction coefficient Ga. Thus, multiplying each of the R value, G value, and B value by the same coefficient is equivalent to increasing or decreasing the amount of light incident on the image sensor of the digital still camera 100. Therefore, by performing exposure correction, it is possible to obtain the same effect as changing the exposure amount in the shooting stage of the digital still camera 100. The exposure correction coefficient Ga is determined by performing the histogram analysis of the reduced image as described above when “automatic exposure correction” is set to ON in the development content designation screen SDC (FIGS. 5 and 6). The On the other hand, when “automatic exposure correction” is set to OFF on the development content designation screen SDC, the exposure correction coefficient Ga is determined according to the setting content of the “exposure” item of the extended setting.

  In step S310, the main development processing unit 350 performs color reproduction processing. Usually, the spectral sensitivity characteristic of the image sensor is different from the spectral sensitivity characteristic of the human eye. Therefore, the color represented by the RGB value of the RAW image is different from the color observed by humans. Therefore, in the color reproduction process, correct colors are reproduced by adjusting the RGB values of the RAW image in accordance with the human visibility characteristics. Note that the color reproduction process is performed by multiplying a 3 × 3 matrix having R, G, and B values as components and a 3 × 3 matrix having a correction coefficient as an element (3 × 3 matrix calculation). Is called. Since this correction coefficient differs for each camera model, the camera model is specified by analyzing the RAW image file format, and a correction coefficient prepared in advance for each model is used.

  In step S312, the development processing unit 350 performs a hue correction process. Similar to the color reproduction process, the hue correction is performed by a 3 × 3 matrix operation. Each element of the 3 × 3 matrix used for hue correction is set based on the target hue rotation angle. The hue rotation angle is determined based on the setting content of the “hue” item set on the development content designation screen SDC (FIGS. 5 and 6).

  In step S314, the main development processing unit 350 performs tone correction processing. In the tone correction process, the tone value of the RAW image is corrected using a tone curve representing the relationship between the input tone value and the output tone value. The tone curve is determined based on the setting content of “style” in the development content designation screen SDC (FIGS. 5 and 6). Further, when the “contrast” of the extended setting is changed, it is determined based on the setting state of “contrast”.

  In step S316, the main development processing unit 350 performs gradation correction (gamma correction) for correcting gradation characteristics. As the gamma value (γ) for performing gamma correction, a value corresponding to the type of image data after development is used. Therefore, the gamma correction corrects the tone characteristics of the RAW image to the tone characteristics corresponding to the type of image data after development.

  In step S318, the main development processing unit 350 performs color conversion processing. This color conversion process is a process of converting the RGB value of the RAW image, which is color information in the RGB space, into the color component value of the color space (for example, YCbCr space) used in the developed image. Similar to the color reproduction process (step S310) and the hue correction process (step S312), the color conversion process is also performed by a 3 × 3 matrix operation.

  In step S320, the main development processing unit 350 performs noise removal processing. The noise removal process is a process for removing a noise component present in an image and generating a clear image. The noise removal process can be performed using, for example, a Gaussian filter.

  In step S322, the main development processing unit 350 performs sharpness adjustment processing. Sharpness adjustment (that is, edge enhancement processing) is processing for correcting and clarifying a portion where an outline in an image is blurred due to the influence of an optical low-pass filter provided in the image sensor. Note that the noise removal process in step S320 and the sharpness adjustment process in step S322 may be performed before the tone correction in step S314.

  In this way, the main development processing unit 350 generates developed image data (printing image data) in JPEG format from an image (developed image) generated by subjecting a RAW image to main development processing. When JPEG format image data is generated as developed image data, 16-bit data used in the calculation process of the main development process is converted to 8 bits when the developed image data is generated. The developed image data format does not necessarily have to be the JPEG format. In general, the developed image data can be in any format as long as it is a standard image data format such as TIFF format or BMP format. After the development of the developed image data, the main development process in FIG. 10 ends, and the control returns to the collective development process in FIG. In step S170, the developed image data generated by the main development process is stored in the developed image storage unit 420 (FIG. 3).

  In step S180, the development processing unit 300 (FIG. 3) determines whether all the RAW image files (target files) designated as the development target have been developed. It is determined that all the target files have been developed. In such a case, the batch development process ends. On the other hand, if it is determined that all the target files have not been developed, that is, there is an undeveloped RAW image file, control is returned to step S110. Then, steps S110 to S180 are repeatedly executed until development of all RAW image files designated as development targets is completed.

  FIG. 11 is a flowchart showing the flow of individual development processing for performing development processing under the development conditions set for each RAW image file. In the individual development process shown in FIG. 11, step S110 is replaced by step S110a, step S122 is provided between step S120 and step S130, and step S150 and step S160 are provided. The collective development shown in FIG. 4 is that the three steps S152 to 156 are provided, the step S162 is provided between the steps S160 and S170, and the step S180 is omitted. It is different from processing. Other points are the same as the batch development processing.

  In step S110a, the development processing unit 300 (FIG. 3) acquires a development designation as in step S110 of FIG. FIG. 12 is an explanatory diagram showing an example of the development content designation screen SDCa displayed in step S110a. The development content designation screen SDCa of the individual development processing shown in FIG. 12 includes a point that the extended setting area REI is omitted and a point that the “development start” button BDS is replaced with the “image selection” button BIS. This is different from the development content designation screen SDC (FIGS. 5 and 6) for development processing. As shown in FIG. 12, on the development content designation screen SDCa for individual development processing, one RAW image file to be subjected to individual development processing is selected in the image selection area. When the user selects a RAW image file and operates the “select image” button BIS, the selected RAW image file is designated as a development target, and the conditions set on the development content designation screen SDCa are designated as development conditions. The In the example of FIG. 12, “automatic exposure compensation” is set to ON, and the style is set to “faithful” to which standard development conditions are applied.

  In step S122, the display development processing unit 340 (FIG. 3) generates a display RAW image for displaying the development result on the display 220 from the RAW image data acquired in step S120. The display RAW image is an image generated by performing pre-gain processing and pixel interpolation processing on the RAW image. The generated display RAW image is stored in the intermediate data storage unit 410 (FIG. 3). The display image generation in step S122 need not be the same as the main development process in step S160. Since the image for normal display is a temporary image that is used only for display, the VGA size (640 × 480 pixels) to XGA size (1024 × 768 pixels) having a lower image quality than the developed image. The image of On the other hand, since a RAW image to be developed has 6 million pixels or 10 million pixels, the image is reduced at a high reduction rate when a display image is generated. For this reason, the values of R, G1, B, and G2 adjacent in the horizontal and vertical directions are used instead of the advanced and high-quality pixel interpolation (step S304) used in the main development in step S160. Simple and high-speed pixel interpolation that generates reduced image data of 1/2 in the vertical and horizontal directions by performing the arithmetic mean, and using the R, B, (G1 + G2) / 2 values as the RGB values of one pixel. To generate a display image.

  In step S152, the display RAW image is subjected to development processing (display development processing), and a display image that is a developed image is generated. FIG. 13 is a flowchart showing the flow of the display development process executed in step S152. The display development process shown in FIG. 13 omits the pre-gain process (step S302) and the pixel interpolation process (step S304) that are performed when the display RAW image is generated (step S122 in FIG. 11). This is different from the main development process shown in FIG. 10 in that an image size adjustment process (step S305) is provided before the white balance adjustment (step S306). The other points are the same as the main development processing.

  In step S305, the display development processing unit 340 adjusts the size of the display RAW image stored in the intermediate data storage unit 410 (image size adjustment). Specifically, the display RAW image is changed to a size suitable for display on the display 220 (for example, 640 × 480 pixels) by reducing or cutting out. In this embodiment, the image size is adjusted in the display development process shown in FIG. 13, but the image size may be adjusted when the display RAW image is generated in step S122 in FIG.

  Next, in steps S306 to S322, the reduced display RAW image is subjected to white balance adjustment processing, exposure correction processing, color reproduction processing, hue correction processing, tone correction processing, as in the main development processing (FIG. 10). A gamma correction process, a color conversion process, a noise removal process, and a sharpness adjustment process are sequentially performed. By applying these processes to the display RAW image, a developed image for display (preview image) is generated. Note that the development conditions acquired in step S110a are used for generating the preview image. After the generation of the preview image, the control is returned to the individual development processing of FIG. 11, and the preview image is displayed on the display 220 in step S154. Thus, since the display RAW image is reduced to the display size in steps S122 and S305, the amount of processing related to steps S306 to S322 for the display image decreases in proportion to the number of pixels. In other words, the processing can be performed at a very high speed, which has a great effect of reducing the waiting time of the user who performs the development processing.

  FIG. 14 is an explanatory diagram showing the development result preview screen SPV displayed in step S154. On the development result preview screen SPV, a preview image IPV representing the result of the development processing according to the contents set in FIG. In the development result preview screen SPV, in addition to the setting area RBI provided on the development content designation screen SDCa in FIG. 12 for setting the development conditions, the development conditions used in the display development process are displayed. A region RDC is provided. In the development condition display area RDC, the setting states of the items “exposure”, “color temperature”, “hue”, “contrast”, and “sharpness” are shown as development conditions. In the development condition display area RDC, a check box indicating whether or not automatic correction has been performed is displayed. In the development content designation screen SDCa shown in FIG. 12, “automatic exposure correction” is set to ON. The result obtained by the analysis for automatic correction is displayed in the development condition display area RDC by default. When it is determined that it is desirable to increase the exposure by two levels and decrease the color temperature by one level, as shown in FIG. 14, a triangular cursor is set at a position where exposure correction is performed to brighten the two-level image, and automatic correction is performed. The check box of is checked. In addition, the cursor is set on the color temperature one step lower temperature side.

  When the user performs an operation on the development result preview screen SPV, the content of the operation is acquired by the development processing unit 300 (FIG. 3). In step S156 of FIG. 11, the development processing unit 300 determines whether the operation on the development result preview screen SPV is an instruction to change the development conditions. If the user operation is an instruction to change the development conditions, the control is returned to step S130, and steps S130 to S156 are repeatedly executed. On the other hand, if the user's operation is not an instruction to change the development condition, that is, if the user operates the “development start” button BDS, the control is moved to step S160.

  FIG. 15 is an explanatory diagram showing an example in which the user changes the development conditions on the development result preview screen SPV in FIG. In the example of FIG. 15, when the user operates the development condition display area RDC, the contrast is set higher by two steps than the state shown in FIG. 14, and the sharpness is set lower by two steps. As shown in FIG. 15, when the setting of the development condition is changed by the user, the display development process is performed again in step S152, and the preview image IPV has a higher contrast than the state shown in FIG. It changes to an image developed again with low sharpness. Further, the check mark attached to the automatic correction check box in the development condition display area RDC is deleted. When the user operates the “development start” button BDS on the development result preview screen SPV in FIG. 15, the main development processing (step S160) is performed on the RAW image.

  In step S162 in FIG. 11, the development processing unit 300 displays the developed image with the size changed as necessary. FIG. 16 shows an example of the developed image display screen SDI displayed on the display 220 (FIG. 3) in step S162. On the developed image display screen SDI, the developed image IDV generated in the main development process (step S160) is displayed in a large size. When the user operates the “save” button BSV provided on the developed image display screen SDI, the developed image data in JPEG format generated from the developed image is stored in the developed image storage unit in step S170 of FIG. 420 (FIG. 3), and the individual development process of FIG. 11 ends.

  FIG. 17 is an explanatory view showing an example different from FIG. 12 of the development content designation screen SDCa displayed in step S110a. The development content designation screen SDCa in FIG. 17 is different from the development content designation screen SDCa in FIG. 12 in that “automatic exposure correction” is set to OFF and the style is set to “automatic style”. Yes. By setting the style to “automatic style”, the development conditions are set based on the scene setting at the time of shooting with the digital still camera 100. The scene setting at the time of shooting is acquired from the shooting conditions stored in the header part of the RAW image file (FIG. 2) (described later). In the example of FIG. 17, it is assumed that no scene is set when shooting with the digital still camera 100. In this case, standard conditions are used as development conditions other than exposure correction.

  Here, the processing method when the style is set to “automatic style” on the development content designation screen SDCa of FIG. 17 will be described. By setting the style to “automatic style”, the development conditions are set based on the scene setting at the time of shooting with the digital still camera 100. The scene setting at the time of shooting is acquired from the shooting conditions stored in the header part of the RAW image file (FIG. 2).

  For example, in the case of a RAW image file conforming to the Exif 2.21 image file format standard for digital still cameras, the shooting scene is recorded in the shooting scene type (SceneCaptureType) recorded in the Exif IFD or Oth IFD Exif Private Tag. ing. The value of the shooting scene type is associated with various shooting scenes. Specifically, a scene of 1 is associated with a scene, 2 is a person, 3 is a night scene, and 0 is a standard scene. The automatic style automatically determines a condition for optimally developing an image using the value of the photographing scene type. When the landscape is set, contrast and saturation are increased and sharpness correction is performed compared to the standard processing. Also, processing such as performing a memory color correction process for the sky and green portions is performed. If a person is set, the memory color of the skin color is corrected, soft sharpness processing suitable for the person and contrast reproduction are performed. In the case of a night view, noise correction is performed on the entire screen without excessive exposure correction, and the process of reproducing the eyes while suppressing the contrast is performed.

  In addition to the shooting scene type (SceneCaptureType), various shooting settings such as shooting contrast (Contrast), shooting saturation (Saturation), and shooting sharpness (Sharpnes) are recorded in the Exif tag. . These photographing contrast values (0 = standard, 1 = soft tone, 2 = high contrast), photographing side values (0 = standard, 1 = low saturation, 2 = high saturation), and photographing sharpness values (0 = Based on the standard, 1 = weak, 2 = strong) and the like, it is possible to change the development conditions during the main development process shown in FIG. 10 and the display development process shown in FIG.

  Further, by recording other settings on the tag as shooting setting information, it is possible to read the tag information and set a more advanced development style (development finishing). For example, “Faith” for more faithful color reproduction, “Soft” for soft image reproduction, “Natural” for creating a natural impression while making pictures more active than faithful, Color temperature before dawn "Cool landscape" with a high-grilled landscape impression, "Warm landscape" with a soft and warm landscape impression at twilight, "Sunset" that vividly reproduces the red of the morning glow and sunset, and a white wedding "Wedding portrait" that softly reproduces the gradation of the dress and the skin of the woman, "Kids" that reproduces the feeling that a naughty child is playing with vivid colors, "Warm room" that expresses warm light indoors Tag information that can be linked to the development conditions, etc., and the development conditions during development may be changed based on the tag information.

  In addition, by recording other settings in the shooting setting information, this tag information is read, and monochrome development and monochrome development conditions that produce contrasts such as red, orange, yellow, and green filters are performed. May be. In the case of monochrome, development processing can be performed by adjusting the luminance signal according to the RGB ratio and making the RGB values of the output images the same. Processing such as applying red, orange, yellow, and green filters can be realized by changing the ratio at the time of generating the luminance value with respect to the R, G, and B values at this time.

  When the user operates the “select image” button BIS on the development content designation screen SDCa of FIG. 17, it is determined in step S130 of FIG. 11 that automatic exposure correction has not been set, and display development processing (without exposure correction) Step S152) is performed. In step S154, a preview image developed without exposure correction is displayed.

  FIG. 18 shows a development result preview screen SPV on which a preview image IPV developed with the setting content on the development content designation screen SDCa shown in FIG. 17 is displayed. In the example of FIG. 18, an underexposed RAW image file is selected in step S110a (FIG. 11). Therefore, an underexposed image is displayed on the development result preview screen SPV as the preview image IPV.

  FIG. 19 is an explanatory diagram illustrating a state where the user has performed an operation of increasing the “exposure” setting by three levels in the state illustrated in FIG. 18. At this time, it is determined in step S156 in FIG. 11 that the development conditions have been changed, and in the display development process (step S152), exposure correction is performed to increase the exposure by three levels. Therefore, as shown in FIG. 19, an image that is slightly overexposed is displayed on the development result preview screen SPV as the preview image IPV.

  FIG. 20 is an explanatory diagram showing a state where the user has performed an operation of changing the setting of “automatic exposure correction” to ON in the state shown in FIG. At this time, in step S156 of FIG. 11, it is determined that the development conditions have been changed. In step S130, it is determined that automatic exposure correction is set, and the exposure correction amount is set based on the analysis result of the reduced image. Thereafter, display development processing (step S152) is performed using the exposure correction amount set based on the analysis result of the reduced image. Therefore, as shown in FIG. 20, an image with appropriate exposure is displayed on the development result preview screen SPV as the preview image IPV.

  FIG. 21 shows a state where the user has performed an operation to set the contrast two steps higher and the sharpness two steps lower in the state shown in FIG. By the user's operation, the preview image IPV changes from the state shown in FIG. 20 to a state with high contrast and low sharpness, as in the example of FIG. Further, the check mark attached to the automatic correction check box in the development condition display area RDC is deleted. When the user operates the “development start” button BDS on the development result preview screen SPV of FIG. 21, the developed image IDV is displayed on the developed image display screen SDI shown in FIG. When the “save” button BSV on the developed image display screen SDI is operated, the developed image data in JPEG format generated from the developed image is stored in the developed image storage unit 420 (FIG. 3).

  FIG. 23 is an explanatory diagram showing still another example of the development content designation screen SDCa displayed in step S110a. The development content designation screen SDCa in FIG. 23 is different from the development content designation screen SDCa in FIG. 17 in that “automatic exposure correction” is set to ON. FIG. 24 is an explanatory diagram showing a development result preview screen SPV displayed when the user operates the “select image” button BIS on the development content designation screen SDCa of FIG. In the development result preview screen SPV shown in FIG. 24, it is shown that the exposure correction for brightening the two-stage image is performed and the color temperature is adjusted to the one-stage low temperature side.

  FIG. 25 is an explanatory diagram illustrating a state in which the user performs an operation of further increasing the “exposure” setting by one level in the state illustrated in FIG. 24. In the example of FIG. 25, since the user changes the setting of “exposure”, “automatic exposure correction” is changed from ON to OFF in FIG. Further, the check mark attached to the automatic correction check box in the development condition display area RDC is deleted.

  FIG. 26 is an explanatory diagram illustrating a state in which the user has changed “style” to “monochrome” in the state illustrated in FIG. 25. In the example of FIG. 26, “style” is changed to “monochrome”. Therefore, a black and white image is displayed as the preview image IPV on the development result preview screen SPV. In addition, the color of the index indicating the values of the “color temperature” and “hue” items that cannot be set in a monochrome image is changed to gray, and these settings cannot be changed.

  FIG. 27 shows a state in which, in the state shown in FIG. 26, the user performs an operation of reducing the exposure correction amount by one step, increasing the contrast by two steps, and reducing the sharpness by two steps. As a result of the change in the development conditions, the preview image IPV is displayed with an image with reduced brightness and higher contrast than the preview image IPV on the development result preview screen SPV in FIG. When the user operates the “development start” button BDS on the development result preview screen SPV of FIG. 27, the black and white developed image IDV is displayed on the developed image display screen SDI shown in FIG. When the “save” button BSV on the developed image display screen SDI is operated, the developed image data in JPEG format generated from the developed image is stored in the developed image storage unit 420 (FIG. 3).

  As described above, in the first embodiment, a reduced image having a predetermined resolution suitable for image analysis is generated from a screen nail image, and the generated reduced image is analyzed to calculate an exposure correction coefficient during development processing. . Then, the brightness of the developed image is corrected by performing development processing using the calculated exposure correction coefficient. For this reason, it is possible to suppress a color shift that may be caused by adjusting the brightness of the developed image that has been subjected to nonlinear processing, and to further improve the brightness of the developed image. Further, depending on the format of the developed image data, the gradation value is represented by 8 bits. In such a case, when the brightness correction is performed on the developed image, the gradation value becomes discrete (referred to as “gradation jump”), and the image after the brightness correction becomes unnatural. there is a possibility. In the first embodiment, the brightness is corrected by a development process in which arithmetic processing is performed using 16-bit data. For this reason, it is possible to suppress the occurrence of gradation jump due to the brightness correction and to make the image on which the brightness correction has been performed more preferable.

  In the first embodiment, the display development process is performed on a display RAW image whose resolution has been reduced in advance. Therefore, the time required for the display development process is shortened, and the time from when the development condition is changed by the user until the preview image is displayed again is shortened. For this reason, the user can quickly confirm the change result of the development condition, so that the change of the development condition becomes easier.

  In the first embodiment, the exposure correction coefficient is calculated from the analysis result of the reduced image. However, other development conditions can be set from the analysis result of the reduced image. For example, it is also possible to detect a white area included in the reduced image and determine a parameter used for white balance adjustment processing (step S306 in FIG. 10) from the color of the white area. It is also possible to detect a specific type of subject (for example, sky or face) included in the reduced image and determine parameters used for the color reproduction process (step S310 in FIG. 10). In this case, the parameters for the color reproduction process are set so that the color of the specific type of subject falls within the target color range.

  Furthermore, it is possible to specify the subject represented by the reduced image and set the development conditions according to the subject. For example, when the subject is a landscape, the development conditions can be determined so as to use a tone curve that increases the contrast of the image in the tone correction process (step S310 in FIG. 10). On the other hand, when the subject is a person, the development conditions can be determined so as to use a tone curve that reduces the contract in the tone correction processing. In addition, when the subject is a night scene, the development conditions can be determined so as to suppress the correction of brightness. Furthermore, the parameters used in the color reproduction process (step S310 in FIG. 10) may be made to perform more suitable color reproduction according to the type (scene) of these subjects.

B. Second embodiment:
FIG. 29 is a flowchart showing the flow of display development processing in the second embodiment. In the display development process of the second embodiment, steps S306 to S312 (white balance adjustment, exposure correction, color reproduction, and hue correction processes) of the display development process of the first embodiment shown in FIG. 13 are corrected. It is replaced with a process matrix generation process (step S330) and a correction process matrix multiplication process (step). The other points are the same as the display development process of the first embodiment.

  As described above, in the white balance adjustment process and the exposure correction process, the RGB value is multiplied by a predetermined correction coefficient. In the color reproduction process and the hue correction process, a 3 × 3 matrix operation is performed. Since these processes are all linear operations, the arithmetic processing performed in steps S306 to S312 of the display development processing in FIG. 13 is expressed by the following equation (3).

  Here, the vector (r, g, b) represents the RGB value before the arithmetic processing performed in steps S306 to S312 (that is, the display RAW image), and the vector (r ′, g ′, b ′). Represents the RGB values after the arithmetic processing performed in steps S306 to S312. The matrix of Expression (3) represents a matrix used for white balance adjustment, exposure correction, color reproduction, and hue correction in order from the right. In this embodiment, each value of the vector (r, g, b) is a 12-bit numerical value (or a numerical value obtained by adding a sign to 12 bits), and each matrix element of the expression (3) is a signed 16-bit value. It is a numerical value.

  In step S330, multiplication of these matrices is performed to generate a single matrix (correction processing matrix). In step S332, the calculation processing of the following equation (4) is performed using the correction processing matrix, whereby the white balance adjustment, exposure correction, color reproduction, and hue correction processing are collectively performed.

  Here, the matrix of Expression (4) is a correction processing matrix obtained by multiplying the matrix of Expression (3). Each element of the matrix of Equation (4) is also 16-bit signed accuracy, and the calculation of the element is performed with 32-bit accuracy so that saturation or accuracy does not decrease in the calculation process from Equation (3), Finally, it is summarized to 16-bit accuracy. The calculation of equation (4) is also performed so that saturation or the like does not occur.

  As described above, by generating a correction processing matrix in advance and performing a plurality of processes at once using the correction processing matrix, it is possible to reduce the amount of calculation processing for the display RAW image.

  In the second embodiment, correction processing matrix generation and batch processing based on the correction processing matrix are applied to display development processing. However, a single matrix can be generated from a plurality of matrices used for processing in advance. If so, it is possible to perform a plurality of processes collectively using the generated single matrix. Specifically, also in the main development processing (FIG. 10), a correction processing matrix is generated instead of steps S306 to S312 (step S330 in FIG. 29), and white balance adjustment, exposure correction, and color are generated using the generated correction processing matrix. Reproduction and hue correction processing may be performed collectively (step S332).

C. Third embodiment:
FIG. 30 is a flowchart showing the flow of the main development process in the third embodiment. In the main development process of the third embodiment, the printing pixel interpolation process (step S304) of the main development process of the first embodiment shown in FIG. 10 is moved after the exposure correction process (step S308). The other points are the same as the main development processing of the first embodiment.

  As described above, in the white balance adjustment process and the exposure correction process, each RGB value is multiplied by a predetermined correction coefficient. For this reason, even in a state where the color information of the pixel is insufficient, the processing can be performed by multiplying the gradation value of the color information possessed by each pixel by the correction coefficient.

  Thus, by performing the white balance adjustment process (step S306) and the exposure correction process (step S308) prior to the pixel interpolation process (step S304), the amount of calculation processing required for these processes can be reduced.

  In the third embodiment, in the display development process (FIG. 13), the pixel interpolation process is performed before the white balance adjustment process (S306). This makes it possible to generate a display RAW image used in the display development process. In the display development process, since the size of the display RAW image generated in advance is small, the amount of calculation processing required for the display development process is smaller than when the white balance adjustment process is performed on the RAW image. Reduced. Further, when receiving a development condition change instruction from the user, it is only necessary to perform the rest of the development process using the interpolated image.

  As described above, in the third embodiment, the execution timing of the pixel interpolation process is set to be different in each of the main development process and the display development process. Therefore, the amount of calculation processing can be reduced in both the main development processing (FIG. 30) and the display development processing (FIG. 13).

D. Fourth embodiment:
FIG. 31 is a schematic configuration diagram schematically showing the configuration of the personal computer 200a in the fourth embodiment. The personal computer 200a of the fourth embodiment differs from the personal computer 200 of the first embodiment shown in FIG. 3 in that the screen nail reduction unit 320 of the development processing unit 300 is replaced with an analysis development processing unit 320a. Yes. Other points are the same as in the first embodiment.

  FIG. 32 is a flowchart showing the flow of batch development processing in the fourth embodiment. The batch development process of the fourth embodiment is different from the batch development process of the first embodiment shown in FIG. 4 in that step S140 is replaced with step S142. The other points are the same as in the first embodiment. In the following, only the batch development process is described, but the same process is performed for the individual development process.

  FIG. 33 is a flowchart showing the flow of analysis development processing executed by the analysis development processing unit 320a in step S142 of FIG. The development process for analysis starts with the main development process shown in FIG. 10 and includes white balance adjustment processing (step S306), exposure correction processing (step S308), hue correction processing (step S312), tone correction processing (step S314), and color conversion. The processing (step S318), noise removal processing (step S320), and sharpness adjustment processing (step S322) are omitted, and the printing pixel interpolation processing (step S304) is reduced RAW image generation processing (step S305). It differs from the main development processing in that it is replaced.

  In step S305, the analysis development processing unit 320a performs simple pixel interpolation processing (simple pixel interpolation) from the RAW image. Then, a reduced RAW image (reduced RAW image) is generated by reducing the image generated by the simple pixel interpolation.

  FIG. 34 is an explanatory diagram showing how simple pixel interpolation is performed. As described above, a Bayer array color filter is arranged on the image sensor of the digital still camera 100 (FIG. 1) as shown in FIG. Therefore, the element that detects green incident light (G pixel) and the element that detects red incident light (R pixel) are in a sparse state as shown in FIGS. 34 (b) and 34 (c). It has become. In the simple pixel interpolation, as shown in FIG. 34 (b), a pixel for which the green color information is missing is obtained by performing an interpolation process by a bilinear method from the G values of four neighboring pixels of the pixel. A value is calculated. In addition, regarding the pixel for which red color information is missing, when the upper and lower two pixels are R pixels, the R value of the pixel is linearly interpolated from the R values of the upper and lower two R pixels. On the other hand, when the diagonal pixel is an R pixel, the R value is calculated from the diagonal four R pixels by the bilinear method. In addition, interpolation processing is performed on pixels lacking blue color information in the same manner as pixels lacking red color information. In the fourth embodiment, the interpolation processing is performed using the bilinear method, but the storage processing may be performed by other methods. For example, the color information missing from a certain pixel can be changed to the color information of any one pixel adjacent to that pixel. In addition, color information that is missing from a certain pixel can be used as an average value of color information of pixels adjacent to that pixel.

  FIG. 35 is an explanatory diagram showing how simple pixel interpolation is performed by a method different from that shown in FIG. FIG. 35A shows a state in which the color filters are arranged according to the Bayer array, as in FIG. 34A. In the simple pixel interpolation of FIG. 35, one pixel (RGB pixel) having all the RGB color information is generated from a 4 × 4 pixel area. Specifically, one R pixel, one G pixel, and one B pixel are selected from the 4 × 4 pixel region. Then, the RGB value of the selected pixel is set to the RGB value of the RGB pixel. In this manner, by generating one RGB pixel from the 4 × 4 pixel area, 1/4 reduction processing can be performed in the simple pixel interpolation. Note that the size of the pixel area used to generate one RGB pixel is not necessarily 4 × 4. The R pixel, the G pixel, and the B pixel (used pixel) in the 4 × 4 pixel region used for generating the RGB pixel may be any pixel as long as it is a pixel in the 4 × 4 pixel region. It can be a pixel. For example, the reduction may be performed in a 2 × 2 area as described in step S122.

  As described above, the image generated by the simple pixel interpolation from the RAW image is reduced to 320 × 240 pixels used for image analysis, and a reduced RAW image is generated.

  The generated reduced RAW image is subjected to color reproduction processing (step S310) and gamma correction processing (S316) to generate a reduced image for analysis. In step S150 of FIG. 32, development conditions are set by analyzing the generated reduced image.

  As described above, in the fourth embodiment, a reduced image is generated by the development process for analysis, and the development condition is set by analyzing the generated reduced image. Therefore, when the RAW image file (FIG. 2) does not contain screen nail image data, or when the screen nail image is not suitable for analysis (for example, when the resolution of the screen nail image is too low or the screen nail image is black and white). In the case of an image), the development conditions can be determined using the reduced image generated by the development process for analysis.

E. Example 5:
FIG. 36 is a schematic configuration diagram schematically showing the configuration of the personal computer 200b in the fifth embodiment. The personal computer 200b of the fifth embodiment is the same as that of the first embodiment shown in FIG. 3 in that a reduced image generation processing selection section 360 and an analysis development processing section 320a are added to the development processing section 300b. Different from the personal computer 200. Other points are the same as in the first embodiment.

  FIG. 37 is a flowchart showing the flow of batch development processing in the fifth embodiment. The batch development process of the fifth embodiment is the same as that of the first embodiment shown in FIG. 4 in that step S132 is added after step S130 and step S142 is added to the branch destination from step S132. This is different from the batch development process. The other points are the same as in the first embodiment. In the following, only the batch development process is described, but the same process is performed for the individual development process.

  In step S132, the reduced image generation processing selection unit 360 determines whether the screen nail image is suitable for analysis. If it is determined that the screen nail image is suitable for analysis, control is transferred to step S140, and the screen nail image is reduced as in the first embodiment to generate a reduced image. On the other hand, if it is determined that the screen nail image is not suitable for analysis, control is transferred to step S142, and a reduced image is generated by the development process for analysis as in the fourth embodiment.

  Whether or not the screen nail image is suitable for analysis is determined based on the data format of the RAW image file (FIG. 2). Note that, as described above, the data format of the RAW image file is specified by information such as the manufacturer and model number stored in the header portion, or the extension attached to the RAW image file. Then, based on the data format of the specified RAW image file, it is determined whether the screen nail image is suitable for analysis. For example, if the resolution of the screen nail image is too low, if the screen nail image is a black and white image, or if the screen nail image data is not included in the RAW image file, the screen nail image is not suitable for analysis. It is judged. Note that the format of the screen nail image data can also be specified by analyzing the screen nail image data itself.

  Thus, in the fifth embodiment, when a screen nail image is suitable for analysis, a reduced image is generated by reducing the screen nail image. On the other hand, when the screen nail image is not suitable for analysis, a reduced image is generated by performing analysis development processing on the RAW image. Normally, the development processing for analysis requires a larger calculation processing amount than the reduction processing of the screen nail image. Therefore, when the screen nail image is suitable for analysis, the screen nail image is reduced and a reduced image is generated, so that the amount of calculation processing for setting the development conditions can be reduced. On the other hand, when the screen nail image is not suitable for analysis, a reduced image is generated by the development process for analysis. Therefore, development conditions can be set more reliably by analyzing reduced images.

  In the fifth embodiment, the image generated by the analysis development process and the screen nail image based on the format of the screen nail image or information on the digital still camera 100 (FIG. 1) such as the manufacturer and model number are used. The reduced image used for the analysis is selected from the reduced images, but the reduced image may be selected according to designation by the user.

F. Example 6:
FIG. 38 is a schematic configuration diagram schematically showing the configuration of the personal computer 200c in the sixth embodiment. The personal computer 200c of the sixth embodiment is different from that shown in FIG. 31 in that a common development condition setting unit 370 is provided in the development processing unit 300c and a common development condition storage unit 430 is provided in the internal storage device 250. This is different from the personal computer 200a of the fourth embodiment shown in FIG. Other points are the same as in the fourth embodiment.

  FIG. 39 is an explanatory diagram showing a common development condition setting screen SCC for setting development conditions (common development conditions) that are commonly set during development processing. This common development condition setting screen SCC is displayed on the display 220 (FIG. 38) by the common development condition setting unit 370 when the user starts the setting application or when the user sets an instruction for setting common development conditions in the image processing application. Is displayed.

  The common development condition setting screen SCC includes an exposure correction amount setting area RCE and a gamma value setting area RCG. When the user moves the triangular index by operating the pointing device in the exposure correction amount setting area RCE, the common setting value of the exposure correction amount is changed. Further, when the user moves the triangular index by operating the pointing device in the gamma value setting area RCG, the common setting value of the gamma value is changed. When the user operates the “set” button BCC, the changed common setting value of the exposure correction amount and the gamma value is stored in the common development condition storage unit 430 by the common development condition setting unit 370. When the user operates the “Cancel” button BCN, the changed common setting value is discarded, and the common development condition stored in the common development condition storage unit 430 is not changed.

  FIG. 40 is a flowchart showing the flow of analysis development processing in the sixth embodiment. In the analysis development process of the sixth embodiment, step S340 for performing reverse exposure correction is added between step S305 and step S310, and step S316 for performing gamma correction is replaced with step S317. This is different from the development processing for analysis of the fourth embodiment. In the following description, only the analysis development process is described, but step S316 for performing gamma correction in the main development process (FIG. 10) and the display development process (FIG. 13) is replaced with step S317.

  In step S340, the analysis development processing unit 320a (FIG. 38) performs reverse exposure correction based on the common setting value of the exposure correction amount stored in the common development condition storage unit 430. Specifically, the RGB value of the reduced RAW image is multiplied by the reciprocal of the exposure correction coefficient corresponding to the common setting value of the exposure correction amount. As described above, the exposure correction coefficient obtained by analyzing the reduced image generated by performing the reverse exposure correction is the standard exposure correction coefficient corresponding to the common setting value of the exposure correction amount. The value multiplied by. Therefore, by performing the main development process or the display development process using the exposure correction coefficient obtained by the analysis, the result of the development process becomes a value reflecting the common setting value of the exposure correction amount set by the user. .

  In the example of FIG. 39, the common setting value of the exposure correction amount is set to “−1/3 EV”. Therefore, the RGB value of the reduced RAW image is multiplied by the reciprocal “1.26” of the exposure correction coefficient “0.79” corresponding to “−1/3 EV”. Therefore, the exposure correction coefficient obtained by analyzing the reduced image is obtained by multiplying the exposure correction coefficient when the exposure correction amount is not set by the exposure correction coefficient “0.79” corresponding to “−1/3 EV”. It will be a thing. Note that “EV” is an exposure value (Exposure Value), and correction of “+1 EV” corresponds to opening the aperture one step or reducing the shutter speed one step. The correction of “−1 EV” corresponds to reducing the aperture by one step or increasing the shutter speed by one step.

  In step S317, the analysis development processing unit 320a performs gamma correction using the common setting value of the gamma value (γ) stored in the common development condition storage unit 430. As described above, also in the main development process (FIG. 10) and the display development process (FIG. 13), step S316 for performing gamma correction is replaced with step S317. Therefore, in gamma correction performed during various development processes, a common setting value set by the user is used as the gamma value (γ) instead of a value corresponding to the type of developed image data. Note that the gamma value is not changed by the analysis of the reduced image. Therefore, the gamma value of the analysis development process, the main development process, and the display development process is set so that the gamma correction result in the analysis development process is the same as the gamma correction result in the main development process and the display development process. A common setting value set by the user is used for.

  As described above, in the sixth embodiment, for the development conditions whose settings are not changed by analysis of the reduced image such as the gamma value, the user sets the common development conditions according to the preference so that the main development process and the display can be performed. The development result of the development process is in accordance with the user's preference. The exposure correction amount for which the correction value is calculated by analyzing the reduced image is reversely corrected at the time of development for analysis. For this reason, the exposure correction coefficient calculated by analyzing the reduced image is changed according to the exposure correction amount set by the user. Therefore, by using the exposure correction coefficient obtained by the analysis for the main development process and the display development process, the development process result reflects the common setting value of the exposure correction amount set according to the user's preference. .

  In the sixth embodiment, the exposure correction amount and the gamma value are set in advance as common development conditions, but other development conditions (for example, color temperature and sharpness) are set as the common development conditions. It is also possible. However, for development conditions whose settings are changed by analysis of a reduced image, reverse correction is performed in analysis development processing for generating a reduced image. As for the development conditions whose settings are not changed by the analysis of the reduced image, the same conditions are used in any of the development processes for analysis, the main development process, and the display development process. In the sixth embodiment, the exposure correction amount and the gamma value are set in advance as the common development conditions, but only one of them may be set as the common development conditions.

G. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

G1. Modification 1:
In each of the above embodiments, the present invention is applied to development processing in a personal computer. However, the present invention can be applied to any apparatus in which development processing is performed. The present invention can be applied to, for example, a device such as a photo viewer that stores an image and displays the stored image, or a printing device that prints an image as long as the device has a development processing function. is there.

1 is a schematic configuration diagram schematically showing a configuration of an image processing system to which an embodiment of the present invention is applied. Explanatory drawing which shows the data format of the RAW image file produced | generated by a digital still camera. The schematic block diagram which shows the structure of a personal computer roughly. 10 is a flowchart showing a flow of batch development processing for performing development processing on a plurality of raw image files under development conditions set in a batch. Explanatory drawing which shows an example of the development content designation | designated screen displayed in step S110. Explanatory drawing which shows an example different from FIG. 5 of the development content designation | designated screen displayed in step S110. The flowchart which shows the flow of the RAW image acquisition process performed in step S120. Explanatory drawing which shows a mode that a reduced image is produced | generated from a screen nail image. Explanatory drawing which shows the mode of the histogram analysis in step S140. 10 is a flowchart showing the flow of the main development process executed in step S150. A flowchart showing the flow of individual development processing for performing development processing under development conditions set for individual RAW image files Explanatory drawing which shows an example of the development content designation | designated screen displayed in step S110a. The flowchart which shows the flow of the developing process for a display performed by step S152. Explanatory drawing which shows the image development result preview screen displayed in step S154. FIG. 15 is an explanatory diagram illustrating an example in which a user changes development conditions on the development result preview screen of FIG. Explanatory drawing which shows an example of the developed image display screen displayed on a display in step S162. Explanatory drawing which shows an example different from FIG. 12 of the development content designation | designated screen displayed in step S110a. FIG. 18 is an explanatory diagram showing a development result preview screen on which a preview image developed with setting contents on the development content designation screen shown in FIG. 17 is displayed. FIG. 19 is an explanatory diagram illustrating a state in which the user performs an operation to increase the “exposure” setting by three levels in the state illustrated in FIG. 18. FIG. 20 is an explanatory diagram illustrating a state in which the user performs an operation of changing the setting of “automatic exposure correction” to ON in the state illustrated in FIG. 19. In the state shown in FIG. 20, explanatory drawing which shows the state which the user performed operation which sets contrast high 2 steps | paragraphs and sets sharpness 2 steps low. FIG. 22 is an explanatory diagram illustrating an example of a developed image display screen displayed on a display when main development is performed in the state illustrated in FIG. 21. Explanatory drawing which shows another example of the development content designation | designated screen displayed in step S110a. FIG. 24 is an explanatory diagram showing a development result preview screen displayed when a user operates an “image selection” button on the development content designation screen of FIG. FIG. 25 is an explanatory diagram illustrating a state in which the user performs an operation of further increasing the “exposure” setting by one level in the state illustrated in FIG. 24. FIG. 26 is an explanatory diagram illustrating a state in which the user changes “style” to “monochrome” in the state illustrated in FIG. 25. FIG. 27 is an explanatory diagram illustrating a state in which the user has performed an operation of decreasing the exposure correction amount by one step and increasing the contrast by two steps in the state illustrated in FIG. 26. FIG. 28 is an explanatory diagram illustrating an example of a developed image display screen displayed on a display when main development is performed in the state illustrated in FIG. 27. 9 is a flowchart showing a flow of display development processing in the second embodiment. 10 is a flowchart showing the flow of a main development process in the third embodiment. The schematic block diagram which shows a structure schematically the structure of the personal computer in 4th Example. 10 is a flowchart showing a flow of batch development processing in a fourth embodiment. 6 is a flowchart showing a flow of analysis development processing executed by an analysis development processing unit. Explanatory drawing which shows a mode that simple pixel interpolation is performed. Explanatory drawing which shows a mode that simple pixel interpolation is performed by the method different from FIG. The schematic block diagram which shows a structure schematically the structure of the personal computer in 5th Example. 10 is a flowchart showing a flow of batch development processing in a fifth embodiment. The schematic block diagram which shows a structure schematically the structure of the personal computer 200c in 6th Example. Explanatory drawing which shows the common development condition setting screen SCC for setting a common development condition. 10 is a flowchart showing the flow of analysis development processing in a sixth embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing system 100 ... Digital still camera 200, 200a, 200b, 200c ... Personal computer 202 ... Bus 210 ... CPU
DESCRIPTION OF SYMBOLS 220 ... Display 230 ... Peripheral device interface 240 ... Memory card interface 250 ... Internal storage device 300, 300b, 300c ... Development processing part 310 ... Raw image acquisition part 320 ... Screen nail reduction part 320a ... Analytical development processing part 330 ... Reduced image Analysis unit 340... Display development processing unit 350... Development processing unit 360. Reduced image generation processing selection unit 370. Common development condition setting unit 410. Intermediate data storage unit 420... Developed image storage unit 430. CV ... Cable MC ... Memory card PRT ... Printer

Claims (8)

A development processing apparatus that executes development processing based on an image file including undeveloped image data generated by a digital camera and accompanying image data that is developed image data attached to the undeveloped image data,
A main development processing unit which generates main developed image data in a predetermined format from the undeveloped image data;
A first analysis image generation unit that generates first analysis image data having a predetermined resolution by performing development processing on the undeveloped image data;
A second analysis image generation unit that generates second analysis image data having a predetermined resolution from the accompanying image data;
A development condition determination unit that determines development conditions used in the main development processing unit by analyzing either the first analysis image data or the second analysis image data;
Based on the priority order between the first analysis image data and the second analysis image data, the analysis image data used for the analysis by the development condition determination unit is converted into the first analysis image data. And an analysis image data selection unit that selects any of the second analysis image data,
A development processing apparatus. The development processing apparatus according to claim 1,
The development image processing apparatus, wherein the analysis image data selection unit determines the priority order based on information about the digital camera included in the image file. The development processing apparatus according to claim 1,
The development image processing apparatus, wherein the analysis image data selection unit determines the priority order based on a type of the accompanying image data. The development processing apparatus according to claim 3,
When the accompanying image data is data representing a black and white image, the analysis image selection unit sets the priority of the first analysis image data higher than the priority of the second analysis image data. A development processing apparatus. The development processing apparatus according to claim 1,
The development condition determination unit
A selection designation obtaining unit for obtaining designation of selection of the image data for analysis by a user;
The priority order is determined based on the designation by the user obtained by the selection designation obtaining unit.
Development processing equipment. A development processing apparatus according to any one of claims 1 to 5,
The development processing apparatus, wherein a resolution of the first analysis image data and a resolution of the second analysis image data are the same. A development processing method for performing development processing based on an image file including undeveloped image data generated by a digital camera and accompanying image data which is developed image data attached to the undeveloped image data,
(A) generating main developed image data in a predetermined format from the undeveloped image data;
(B) generating first analysis image data having a predetermined resolution by performing development processing on the undeveloped image data;
(C) generating second analysis image data having a predetermined resolution from the accompanying image data;
(D) determining the development conditions used in the step (a) by analyzing either the first analysis image data or the second analysis image data;
(E) Based on the priority order between the first analysis image data and the second analysis image data, the analysis image data used for the analysis in the step (d) is changed to the first analysis image data. A step of selecting one of the analysis image data and the second analysis image data;
A development processing method comprising: A computer program for executing development processing based on an image file including undeveloped image data generated by a digital camera and accompanying image data which is developed image data attached to the undeveloped image data. ,
(A) a function of generating main developed image data in a predetermined format from the undeveloped image data;
(B) a function of generating first analysis image data having a predetermined resolution by performing development processing on the undeveloped image data;
(C) a function of generating second analysis image data having a predetermined resolution from the accompanying image data;
(D) a function of determining development conditions used in the function (a) by analyzing either the first analysis image data or the second analysis image data;
(E) Based on the priority order between the first analysis image data and the second analysis image data, the analysis image data used for the analysis by the function (d) is changed to the first analysis image data. A function of selecting from either the analysis image data or the second analysis image data;
A computer program that causes a computer to realize

Images