JP4544319B2 - Image processing apparatus, method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus and method, and more particularly to an apparatus and method relating to image storage and reproduction in a digital input device, and a computer program for realizing the apparatus and method.

  The image processing apparatus disclosed in Patent Document 1 creates a standard image and a non-standard image from a plurality of image data obtained by imaging the same subject multiple times with different exposure amounts, and is necessary for expanding the dynamic range from the non-standard image It is characterized by distinguishing a region and compressing and storing the portion.

Patent Documents 2 to 4 propose a method for recording extended color gamut information in order to realize image reproduction in a color space having a wider color gamut than the standard color space represented by sRGB. Has been. That is, difference information between a restricted color gamut digital image data having a color value in a color space having a restricted color gamut and an extended color gamut digital image having a color value outside the restricted color gamut is used as a restricted color gamut digital image. Recorded in association with data.
JP-A-8-256303 US Pat. No. 6,282,311 US Pat. No. 6,823,312 US Pat. No. 6,282,313

  In a general digital still camera, gradation design is made based on photoelectric conversion characteristics defined in CCIR Rec709. By doing so, the image design is made with the goal of providing a good image when reproduced in the sRGB color space, which is a substantially standard color space of a display for a personal computer (PC).

  On the other hand, in a real scene, the brightness range may be 1: 100 or may exceed 1: 10000, such as sunny days, cloudy days, and nights. Conventional CCD image sensors used in the past cannot acquire information from such a wide luminance range at a time. Therefore, automatic exposure (AE) control is used to determine the optimum luminance cutout range, convert it to an electrical signal according to the previously defined photoelectric conversion characteristics, and reproduce the image on a display such as a CRT. Yes. Alternatively, as disclosed in Patent Document 1, a wide dynamic range is secured by photographing a plurality of images while changing the exposure. However, this multiple exposure technique can only be applied to photographing a stationary object.

  By the way, exposure is mainly used for special subjects or shooting situations such as shooting bridal gowns (white wedding dresses), shooting objects with metallic luster such as automobiles, shooting with close-up strobes, and even backlighting. It is difficult to appropriately match the subject, and a high-quality image that covers a wide luminance range cannot be obtained. For such scenes, it is better to use a system that corrects the captured image later (in the printing process), records the image with a wider dynamic range, and creates an optimal image during printing based on that information. Often a convenient image is obtained.

  However, even in such a case, there is a problem that sufficient image quality cannot be obtained from image information with a limited dynamic range as in the present situation.

  The present invention has been made in view of such circumstances. For normal PC output or the like, an image with a predetermined dynamic range is displayed, and for special applications such as desktop publishing printing applications, as necessary. Another object of the present invention is to provide an image processing apparatus, method, and program capable of creating an optimal image by image processing from information obtained by imaging with a wider dynamic range.

The image processing apparatus according to the present invention in order to achieve the object, a recording unit in which the first image information file consisting of the high-sensitivity image information formed by the pixel of the relatively high sensitivity is recorded, images And a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels associated with the first image information file and the image display means for displaying and outputting the image When recorded on the means, the first image created from the first image information file is displayed on the image display means, and the first image is displayed by the second image information file . And display control means for highlighting an image portion whose reproduction area is expanded on the display screen of the first image.

A first image created from the first image information file displayed on the image display unit, whether the information on the first images there is a difference of the second image information file and the first image information file If there is a difference, the corresponding part is highlighted, such as blinking, framed framed display, brightness (shading) change, and color tone change.

According to another aspect of the present invention, the high-sensitivity image information in the first image information file and the low-sensitivity image information in the second image information file are recorded with different bit depths. It is characterized by that.

According to another aspect of the present invention, the second image information file is characterized in that it is compressed is recorded in the first image information file and different compression scheme.
The image processing method of the present invention also includes a reading step of reading from the recording means a first image information file comprising high-sensitivity image information formed by relatively high-sensitivity pixels, and an image for outputting the image to a display device. When a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels correlated with the first image information file is recorded in the recording means. Is an image portion in which a first image created from the first image information file is displayed on the display device, and a reproduction area is expanded by the second image information file with respect to the first image. And a display control step of highlighting on the display screen of the first image.
According to another aspect of the present invention, the reading step includes high-sensitivity image information in the first image information file and low in the second image information file recorded at different bit depths. Sensitivity image information is read out.
Furthermore, according to another aspect of the present invention, the reading step reads the first image information file and the second image information file that are compressed and recorded by different compression methods. To do.
In addition, the image processing program of the present invention includes a read function for reading out a first image information file composed of high-sensitivity image information formed by relatively high-sensitivity pixels from a recording unit, and an image for outputting an image to a display device. When a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels correlated with the display function and the first image information file is recorded in the recording unit Is an image portion in which a first image created from the first image information file is displayed on the display device, and a reproduction area is expanded by the second image information file with respect to the first image. And a display control function for emphasizing the image on the display screen of the first image.
According to another aspect of the present invention, the read function has high sensitivity image information in the first image information file and low in the second image information file, which are recorded at different bit depths. Sensitivity image information is read out.
Furthermore, according to another aspect of the present invention, the reading function reads the first image information file and the second image information file that are compressed and recorded by different compression methods. To do.

  The image processing apparatus of the present invention can be mounted on an electronic camera such as a digital camera or a video camera, or can be realized by a computer. A program for realizing each means constituting the above-described image processing apparatus by a computer is recorded on a CD-ROM, a magnetic disk or other recording medium, and the program is provided to a third party through the recording medium, or the Internet It is also possible to provide a download service for the program through a communication line.

  According to the present invention, the user can grasp the extensibility of image reproduction by visualizing a region portion that can be reproduced in more detail using the associated extended information.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Image sensor structure]
First, the structure of an image sensor for wide dynamic range imaging used in an electronic camera to which the present invention is applied will be described. FIG. 1 is a plan view showing an example of the structure of the light receiving surface of the CCD 20. FIG. 1 shows a state in which two light receiving cells (pixels PIX) are arranged side by side, but in actuality, a large number of pixels PIX are arranged at a constant arrangement period in the horizontal direction (row direction) and the vertical direction (column direction). Has been.

  Each pixel PIX includes two photodiode regions 21 and 22 having different sensitivities. The first photodiode region 21 has a relatively large area, and constitutes a main photosensitive portion (hereinafter referred to as a main photosensitive pixel). The second photodiode region 22 has a relatively small area and forms a secondary photosensitive portion (hereinafter referred to as a secondary photosensitive pixel). A vertical transfer path (VCCD) 23 is formed on the right side of the pixel PIX.

  The configuration shown in FIG. 1 is a pixel arrangement having a honeycomb structure, and pixels (not shown) are arranged at positions shifted by a half pitch in the horizontal direction above and below the two illustrated pixels PIX. A vertical transfer path 23 shown on the left side of each pixel PIX shown in FIG. 1 is for reading out and transferring charges from pixels (not shown) arranged above and below the image PIX. is there.

  As shown by the dotted lines in FIG. 1, transfer electrodes 24, 25, 26 and 27 (collectively indicated by EL) necessary for four-phase driving (φ1, φ2, φ3, φ4) are arranged above the vertical transfer path 23. Is done. For example, when the transfer electrode is formed of two-layer polysilicon, the first transfer electrode 24 to which the pulse voltage of φ1 is applied and the third transfer electrode 26 to which the pulse voltage of φ3 is applied are the first layer polysilicon. A second transfer electrode 25 formed of a silicon layer, to which a pulse voltage of φ2 is applied, and a fourth transfer electrode 27 to which a pulse voltage of φ4 is applied are formed of a second polysilicon layer. The transfer electrode 24 also controls charge reading from the secondary photosensitive pixel 22 to the vertical transfer path 23. The transfer electrode 25 also controls charge reading from the main photosensitive pixel 21 to the vertical transfer path 23.

2 is a cross-sectional view taken along line 2-2 in FIG. 1, and FIG. 3 is a cross-sectional view taken along line 3-3 in FIG. As shown in FIG. 2, a p-type well 31 is formed on one surface of the n-type semiconductor substrate 30. Two n-type regions 33 and 34 are formed in the surface region of the p-type well 31 to constitute a photodiode. A photodiode in the n-type region indicated by reference numeral 33 corresponds to the main photosensitive pixel 21, and a photodiode in the n-type region indicated by reference numeral 34 corresponds to the secondary photosensitive pixel 22. The p ⁺ -type region 36 is a channel stop region that performs electrical separation of the pixel PIX, the vertical transfer path 23, and the like.

  As shown in FIG. 3, an n-type region 37 constituting the vertical transfer path 23 is arranged in the vicinity of the n-type region 33 constituting the photodiode. The p-type well 31 between the n-type regions 33 and 37 constitutes a read transistor.

  An insulating layer such as a silicon oxide film is formed on the surface of the semiconductor substrate, and a transfer electrode EL made of polysilicon is formed thereon. The transfer electrode EL is disposed so as to cover the vertical transfer path 23. An insulating layer such as silicon oxide is further formed on the transfer electrode EL, and a light shielding film 38 that covers the components such as the vertical transfer path 23 and has an opening above the photodiode is formed of tungsten or the like. .

  An interlayer insulating film 39 made of phosphosilicate glass or the like is formed so as to cover the light shielding film 38, and the surface thereof is flattened. A color filter layer (on-chip color filter) 40 is formed on the interlayer insulating film 39. The color filter layer 40 includes, for example, three or more color areas such as a red area, a green area, and a blue area, and one color area is assigned to each pixel PIX.

  A microlens (on-chip microlens) 41 corresponding to each pixel PIX is formed on the color filter layer 40 using a resist material or the like. One microlens 41 is formed on each pixel PIX and has a function of converging light incident from above into an opening defined by the light shielding film 38.

  Light incident through the microlens 41 is color-separated by the color filter layer 40 and is incident on each photodiode region of the main photosensitive pixel 21 and the secondary photosensitive pixel 22. The light incident on each photodiode region is converted into signal charges corresponding to the amount of light, and read out separately to the vertical transfer path 23.

  In this way, two types of image signals (high-sensitivity image signal and low-sensitivity image signal) having different sensitivities can be separately extracted from one pixel PIX, and an optically in-phase image signal is obtained.

  FIG. 4 shows the arrangement of the pixels PIX and the vertical transfer paths 23 in the light receiving area PS of the CCD 20. The pixel PIX has a honeycomb structure in which the center points of the geometrical shape of the cells are arranged so as to be shifted by half the pixel pitch (1/2 pitch) every other row direction and column direction. That is, in the rows (or columns) of pixels PIX adjacent to each other, the cell arrangement in one row (or column) is in the row direction (or column direction) with respect to the cell arrangement in the other row (or column). The structure is arranged so as to be relatively shifted by approximately ½ of the arrangement interval.

  In FIG. 4, a VCCD driving circuit 44 for applying a pulse voltage to the transfer electrode EL is disposed on the right side of the light receiving region PS in which the pixels PIX are arranged. Each pixel PIX includes the main photosensitive pixel 21 and the secondary photosensitive pixel 22 as described above. The vertical transfer path 23 is arranged meandering close to each column.

  Further, a horizontal transfer path (HCCD) 45 that transfers the signal charges transferred from the vertical transfer path 23 in the horizontal direction is provided below the light receiving area PS (the lower end side of the vertical transfer path 23).

  The horizontal transfer path 45 is constituted by a transfer CCD driven by two phases, and the final stage (the leftmost stage in FIG. 4) of the horizontal transfer path 45 is connected to the output unit 46. The output unit 46 includes an output amplifier, performs charge detection of the input signal charge, and outputs it as a signal voltage to the output terminal. In this way, a signal photoelectrically converted by each pixel PIX is output as a dot-sequential signal sequence.

  FIG. 5 shows another structural example of the CCD 20. 5 is a plan view, and FIG. 6 is a cross-sectional view taken along line 6-6 of FIG. In these drawings, members that are the same as or similar to those shown in FIGS. 1 and 2 are given the same reference numerals, and descriptions thereof are omitted.

As shown in FIGS. 5 and 6, and p ⁺ -type isolation region 48 between the main photosensitive pixel 21 and従感light pixels 22 is formed. The isolation region 48 functions as a channel stop region (channel stopper), and electrically isolates the photodiode region. A light shielding film 49 is formed above the separation region 48 at a position corresponding to the separation region 48.

  By using the light shielding film 49 and the separation region 48, the incident light is efficiently separated and the charges accumulated in the main photosensitive pixel 21 and the secondary photosensitive pixel 22 are prevented from being mixed thereafter. Other configurations are the same as those of the example shown in FIGS.

  Further, the cell shape and the opening shape of the pixel PIX are not limited to the examples shown in FIGS. 1 and 5 and can take various forms such as a polygon and a circle. Further, the separation shape (divided form) of each light receiving cell is not limited to the shape shown in FIG. 1 or FIG.

  FIG. 7 shows still another structural example of the CCD 20. In FIG. 7, the same or similar members as those in the example shown in FIG. 1 and FIG. FIG. 7 shows a configuration in which the two photosensitive portions (21, 22) are separated in an oblique direction.

  In this way, it is sufficient if the charges accumulated in each divided photosensitive area can be read out separately to the vertical transfer path, and the division shape, the number of divisions, the size relationship of the areas, etc. are appropriately designed. However, the area of the secondary photosensitive pixel is set to a smaller value than the area of the main photosensitive pixel. In addition, it is preferable to suppress a decrease in the area of the main photosensitive portion and minimize a decrease in sensitivity.

  FIG. 8 is a graph showing the photoelectric conversion characteristics of the main photosensitive pixel 21 and the secondary photosensitive pixel 22. The horizontal axis represents the amount of incident light, and the vertical axis represents the image data value (QL value) after A / D conversion. In this example, 12-bit data is illustrated, but the number of bits is not limited to this.

  As shown in the figure, the sensitivity ratio between the main photosensitive pixel 21 and the secondary photosensitive pixel 22 is 1: 1 / a (where a> 1, a = 16 in this example). The output of the main photosensitive pixel 21 gradually increases in proportion to the amount of incident light. When the amount of incident light is “c”, the output reaches a saturation value (QL value = 4095). Thereafter, even if the amount of incident light increases, the output of the main photosensitive pixel 21 becomes constant. This “c” will be referred to as the saturation light amount of the main photosensitive pixel 21.

  On the other hand, the sensitivity of the secondary photosensitive pixel 22 is 1 / a of the sensitivity of the main photosensitive pixel 21 and is saturated at a QL value = 4095 / b when the incident light quantity is α × c (where b> 1, α = a / b, in this example b = 4, α = 4). “Α × c” at this time is referred to as the saturation light amount of the secondary photosensitive pixel 22.

  In this way, by combining the main photosensitive pixel 21 and the secondary photosensitive pixel 22 having different sensitivities and saturations, the dynamic range of the CCD 20 can be expanded by a factor of α compared to the configuration of only the main photosensitive pixel. In this example, the dynamic range is expanded about four times with a sensitivity ratio of 1/16 and a saturation ratio of 1/4. When the maximum dynamic range when only the main photosensitive pixels are used is 100%, in this example, the dynamic range is expanded up to about 400% by utilizing the secondary photosensitive pixels.

  As described above, in an image sensor such as a CCD, light received by a photodiode through a color filter such as RGB or C (cyan), M (magenta), and Y (yellow) is converted into the signal. Of this, how much light information is obtained depends on the optical system including the lens, CCD sensitivity, and saturation. For elements with relatively high sensitivity but a small amount of charge that can be stored, and elements with relatively low sensitivity but a large amount of charge that can be stored, the latter is more intense than the incident light. However, an appropriate signal can be provided, and the dynamic range is wide.

  As means for setting how to respond to the intensity of light, (1) adjust the amount of light entering the photodiode, (2) change the amplifier gain of the source follower that receives light and changes it to voltage -There exist aspects, such as. In the case of (1), it can be adjusted with respect to the photodiode by the light transmission characteristics and relative positional relationship of the microlens in the upper layer portion. On the other hand, the amount of charge that can be accumulated is determined by the size of the photodiode and the like. As described with reference to FIGS. 1 to 7, by arranging two photodiodes (21, 22) having different sizes, it is possible to obtain a signal that can respond to a contrast ratio of different light. By adjusting the sensitivity of the two photodiodes (21, 22), it is possible to finally realize an imaging device (CCD 20) having a wide dynamic range.

[Example of camera capable of wide dynamic range imaging]
Next, an electronic camera equipped with the above-described CCD for wide dynamic range imaging will be described.

  FIG. 9 is a block diagram showing the configuration of the electronic camera according to the embodiment of the present invention. The camera 50 is a digital camera that converts an optical image of a subject imaged through the CCD 20 into digital image data and records it on a recording medium 52. The camera 50 includes a display unit 54, and can display a video being captured and a reproduced image of recorded image data on the display unit 54.

  The overall operation of the camera 50 is centrally controlled by a central processing unit (CPU) 56 built in the camera. The CPU 56 functions as a control means for controlling the camera system according to a predetermined program, and performs various calculations such as automatic exposure (AE) calculation, automatic focus adjustment (AF) calculation, and auto white balance (AWB) control. Functions as a means.

  The CPU 56 is connected to a ROM 60 and a memory (RAM) 62 via a bus (not shown). The ROM 60 stores programs executed by the CPU 56 and various data necessary for control. The memory 62 is used as a program development area and a calculation work area for the CPU 56, and is also used as a temporary storage area for image data.

  As a temporary storage area for image data, the memory 62 is obtained from a first area (hereinafter referred to as a first image memory) 62A that mainly stores image data obtained from the main photosensitive pixel 21, and is obtained mainly from the subordinate photosensitive pixel 22. And a second area (hereinafter referred to as a second image memory) 62B for storing image data.

  In addition, an EEPROM 64 is connected to the CPU 56. The EEPROM 64 is a nonvolatile storage means for storing defective pixel information of the CCD 20, data necessary for control of AE, AF, AWB, etc., or customization information set by the user, and data can be rewritten as necessary. In addition, the information content is retained even when the power is turned off. The CPU 56 performs calculations and the like with reference to the data in the EEPROM 64 as necessary.

  The camera 50 is provided with an operation unit 66 for a user to input various commands. The operation unit 66 includes various operation units such as a shutter button, a zoom switch, and a mode switch. The shutter button is an operation means for inputting an instruction to start photographing, and is composed of a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. When S1 is on, AE and AF processing are performed, and when S2 is on, recording exposure is performed. The zoom switch is an operation means for changing the photographing magnification and the reproduction magnification. The mode switch is an operation means for switching between the shooting mode and the playback mode.

  In addition to the above, the operation unit 66 has a shooting mode setting for setting an optimum operation mode (continuous shooting mode, auto shooting mode, manual shooting mode, portrait mode, landscape mode, night view mode, etc.) according to the shooting purpose. Means, a menu button for displaying a menu screen on the display unit 54, a cross button (cursor moving operation means) for selecting a desired item from the menu screen, an OK button for instructing selection confirmation and execution of processing, a selection item, etc. Cancel button, cancel instruction contents, cancel button for inputting command to return to previous operation state, ON / OFF of display unit 54, switching of display method, display / display of on-screen display (OSD) Display button for performing non-display switching, etc., whether to implement dynamic range expansion processing (image composition) Operation means such as a D-range extension mode switch for-option also included.

  Note that the operation unit 66 is not limited to a push-type switch member, dial member, lever switch, or the like, and includes those realized by a user interface for selecting a desired item from a menu screen. It is.

  A signal from the operation unit 66 is input to the CPU 56. The CPU 56 controls each circuit of the camera 50 based on an input signal from the operation unit 66. For example, lens driving control, photographing operation control, charge readout control from the CCD 20, image processing control, image data recording / reproduction control, File management in the recording medium 52, display control of the display unit 54, and the like are performed.

  For the display unit 54, for example, a color liquid crystal display is used. Instead of the liquid crystal display, other types of display devices (display means) such as an organic EL may be used. The display unit 54 can be used as an electronic viewfinder for checking the angle of view at the time of shooting, and is used as a means for reproducing and displaying a recorded image. The display unit 54 is also used as a user interface display screen, and displays information such as menu information, selection items, and setting contents as necessary.

  Next, the shooting function of the camera 50 will be described.

  The camera 50 includes an optical system unit 68 and a CCD 20. Instead of the CCD 20, it is also possible to use another type of image sensor such as a MOS type solid-state image sensor. The optical system unit 68 includes a photographing lens (not shown) and a mechanical shutter mechanism that also serves as an aperture. The photographic lens is composed of an electric zoom lens, and a detailed optical configuration is not shown, but it mainly contributes to focus adjustment and a variable power lens group and a correction lens group that cause a magnification change (focal length variable) action. Including a focus lens.

  When the photographer operates the zoom switch of the operation unit 66, an optical system control signal is output from the CPU 56 to the motor drive circuit 70 in accordance with the switch operation. The motor driving circuit 70 generates a lens driving signal based on a control signal from the CPU 56 and supplies it to a zoom motor (not shown). Thus, the zoom motor is operated by the motor drive voltage output from the motor drive circuit 70, and the variable power lens group and the correction lens group in the photographic lens move back and forth along the optical axis, so that the focal length ( The optical zoom magnification is changed.

  The light that has passed through the optical system unit 68 enters the light receiving surface of the CCD 20. A large number of photo sensors (light receiving elements) are arranged in a plane on the light receiving surface of the CCD 20, and primary color filters of red (R), green (G), and blue (B) are arranged in a predetermined arrangement corresponding to each photo sensor. Arranged in structure. A color filter such as CMY can be used instead of the RGB color filter.

  The subject image formed on the light receiving surface of the CCD 20 is converted into a signal charge of an amount corresponding to the amount of incident light by each photosensor. The CCD 20 has an electronic shutter function for controlling the charge accumulation time (shutter speed) of each photosensor according to the timing of the shutter gate pulse.

  The signal charge accumulated in each photosensor of the CCD 20 is a voltage signal (image signal) corresponding to the signal charge based on pulses (horizontal drive pulse φH, vertical drive pulse φV, overflow drain pulse) given from the CCD driver 72. Read sequentially. The image signal output from the CCD 20 is sent to the analog processing unit 74. The analog processing unit 74 is a processing unit including a CDS (correlated double sampling) circuit and a GCA (gain control amplifier) circuit. The analog processing unit 74 performs color separation into sampling processing and R, G, and B color signals. Then, the signal level of each color signal is adjusted.

  The image signal output from the analog processing unit 74 is converted into a digital signal by the A / D converter 76 and then stored in the memory 62 via the signal processing unit 80. The timing generator (TG) 82 gives timing signals to the CCD driver 72, the analog processing unit 74 and the A / D converter 76 in accordance with instructions from the CPU 56, and the circuits are synchronized by this timing signal. Yes.

  The signal processing unit 80 is a digital signal processing block that also serves as a memory controller that controls reading and writing of the memory 62. The signal processing unit 80 interpolates the spatial shift of the color signal associated with the color filter array of the single-chip CCD, an auto calculation unit that performs AE / AF / AWB processing, a white balance circuit, a gamma conversion circuit, and a synchronization circuit. Processing circuit for calculating the color of each point), luminance / color difference signal luminance / color difference signal generation circuit, contour correction circuit, contrast correction circuit, compression / decompression circuit, display signal generation circuit, etc. The image signal is processed using the memory 62 in accordance with the command.

  The data (CCDRAW data) stored in the memory 62 is sent to the signal processing unit 80 via the bus. Although details of the signal processing unit 80 will be described later, the image data sent to the signal processing unit 80 is subjected to white balance adjustment processing, gamma conversion processing, luminance signal (Y signal), and color difference signals (Cr, Cb signals). After predetermined signal processing such as conversion processing (YC processing) is performed, the data is stored in the memory 62.

  When the captured image is output to the display unit 54 on the monitor, the image data is read from the memory 62 and sent to the display conversion circuit of the signal processing unit 80. The image data sent to the display conversion circuit is converted into a predetermined display signal (for example, an NTSC color composite video signal) and then output to the display unit 54. The image data in the memory 62 is periodically rewritten by the image signal output from the CCD 20, and the video signal generated from the image data is supplied to the display unit 54, so that the video being captured (through image) can be obtained. It is displayed on the display unit 54 in real time. The photographer can confirm the angle of view (composition) by the through image displayed on the display unit 54.

  When the photographer determines the angle of view and presses the shutter button, the CPU 56 detects this and performs shooting preparation operations such as AE processing and AF processing in response to half-pressing of the shutter button (S1 = ON). In response to full depression of (S2 = ON), CCD exposure and readout control for capturing a recording image is started.

  That is, the CPU 56 performs various calculations such as a focus evaluation calculation and an AE calculation from the image data fetched in response to S1 = ON, and sends a control signal to the motor drive circuit 70 based on the calculation result. The AF motor is controlled to move the focus lens in the optical system unit 68 to the in-focus position.

  The AE calculation unit of the auto calculation unit includes a circuit that divides one screen of a captured image into a plurality of areas (for example, 8 × 8) and integrates RGB signals for each divided area, and provides the integrated value to the CPU 56. To do. The integrated value may be obtained for each of the RGB color signals, or the integrated value may be obtained for only one of these colors (for example, the G signal).

  The CPU 56 performs weighted addition based on the integrated value obtained from the AE calculation unit, detects the brightness of the subject (subject brightness), and calculates an exposure value (shooting EV value) suitable for shooting.

  The AE of the camera 50 performs light measurement a plurality of times in order to accurately measure a wide luminance range and correctly recognizes the luminance of the subject. For example, when the range of 5 to 17 EV is measured, and the range of 3 EV can be measured by one photometry, the photometry is performed four times at maximum while changing the exposure condition.

  Metering is performed under certain exposure conditions, and the integrated value of each divided area is monitored. If there is a saturated area in the image, photometry is performed with different exposure conditions. On the other hand, if there is no saturated area in the image, metering can be performed correctly under that exposure condition, and therefore the exposure condition is not changed further.

  In this way, photometry is performed in a plurality of times to measure the wide range (5 to 17 EV), and the optimum exposure condition is determined. Note that the range that can be measured by one photometry and the range that should be measured can be appropriately designed for each camera model.

  The CPU 56 controls the aperture and the shutter speed based on the above AE calculation result, and acquires a recording image in response to S2 = ON. The camera 50 of this example reads data from only the main photosensitive pixel 21 during the through image, and creates an image for the through image from the image signal of the main photosensitive pixel 21. Further, the AE process and the AF process associated with S1 = ON of the shutter button are performed based on a signal obtained from the main photosensitive pixel 21. When a shooting mode for performing wide dynamic range imaging is selected, or when a wide dynamic range imaging mode is automatically selected based on an AE result (ISO sensitivity or photometric value) or a white balance gain value The exposure of the CCD 20 is performed in response to S2 = ON of the shutter button. After exposure, the main photosensitive pixel is first synchronized with the vertical drive signal (VD) in a state in which the mechanical shutter is closed to block the entrance of light. Then, the charge of the sub-photosensitive pixel 22 is read out.

  The camera 50 has a strobe device 84. The strobe device 84 is a block including a discharge tube (for example, a xenon tube) as a light emitting unit, a trigger circuit, a main capacitor for storing discharge energy, and a charging circuit thereof. The CPU 56 sends a command to the strobe device 84 as necessary to control the light emission of the strobe device 84.

  Thus, the image data taken in response to the full press of the shutter button (S2 = ON) is subjected to YC processing and other predetermined signal processing in the signal processing unit 80, and then, a predetermined compression format (for example, JPEG format). And is recorded on the recording medium 52 via a media interface unit (not shown in FIG. 9). The compression format is not limited to JPEG, and MPEG or other methods may be adopted.

  As a means for storing image data, various media such as a semiconductor memory card represented by SmartMedia (trademark), CompactFlash (registered trademark), a magnetic disk, an optical disk, and a magneto-optical disk can be used. Further, the recording medium (internal memory) built in the camera 50 is not limited to the removable medium.

  When the playback mode is selected by the mode selection switch of the operation unit 66, the last image file (last recorded file) recorded on the recording medium 52 is read. The image file data read from the recording medium 52 is decompressed by the compression / decompression circuit of the signal processing unit 80, converted into a display signal, and output to the display unit 54.

  By operating the cross button during single-frame playback in the playback mode, it is possible to move forward or backward in the forward direction, the next file that has been forwarded forward is read from the recording medium 52, and the display image is updated. .

  FIG. 10 is a block diagram showing a signal processing flow in the signal processing unit 80 shown in FIG.

  As shown in FIG. 10, main photosensitive pixel data (referred to as high-sensitivity image data) converted into a digital signal by the A / D converter 76 is subjected to an offset process in an offset processing circuit 91. The offset processing circuit 91 is a processing unit that corrects the dark current component of the CCD output, and performs an operation of subtracting the value of the optical black (OB) signal obtained from the light-shielded pixel on the CCD 20 from the pixel value. Data (high sensitivity RAW data) output from the offset processing circuit 91 is sent to the linear matrix circuit 92.

  The linear matrix circuit 92 is a color tone correction processing unit that corrects the spectral characteristics of the CCD 20. The data corrected in the linear matrix circuit 92 is sent to a white balance (WB) gain adjustment circuit 93. The WB gain adjustment circuit 93 includes a gain variable amplifier for increasing / decreasing the levels of the R, G, and B color signals, and adjusts the gain of each color signal based on a command from the CPU 56. The signal subjected to the white balance adjustment processing in the WB gain adjustment circuit 93 is sent to the gamma correction circuit 94.

  The gamma correction circuit 94 converts the input / output characteristics so as to obtain a desired gamma characteristic in accordance with a command from the CPU 56. The image data subjected to gamma correction in the gamma correction circuit 94 is sent to the synchronization processing circuit 95.

  The synchronization processing circuit 95 interpolates the spatial shift of the color signal associated with the color filter array structure of the single CCD 20 and calculates the color (RGB) of each point, and the luminance (Y) signal from the RGB signal. And a YC conversion processing unit for generating color difference signals (Cr, Cb). The luminance / color difference signal (YCr Cb) generated by the synchronization processing circuit 95 is sent to various correction circuits 96.

  The various correction circuits 96 include, for example, a contour enhancement (aperture correction) circuit, a color correction circuit using a color difference matrix, and the like. The image data that has undergone the required correction processing in the various correction circuits 96 is sent to the JPEG compression circuit 97. The image data compressed by the JPEG compression circuit 97 is recorded on the recording medium 52 as an image file.

  Similarly, sub-photosensitive pixel data (referred to as low-sensitivity image data) converted into a digital signal by the A / D converter 76 is subjected to offset processing in the offset processing circuit 101. Data (low sensitivity RAW data) output from the offset processing circuit 101 is sent to the linear matrix circuit 102.

  The data output from the linear matrix circuit 102 is sent to the white balance (WB) gain adjustment circuit 103, and white balance adjustment is performed. The signal subjected to the white balance adjustment processing is sent to the gamma correction circuit 104.

  The low-sensitivity image data output from the low-sensitivity image data linear matrix circuit 102 is also supplied to the integration circuit 105. The integration circuit 105 divides the imaging screen into a plurality of areas (for example, 16 × 16), performs a process of integrating the R, G, and B pixel values for each area, and calculates an average value for each color. calculate.

  Among the average values calculated by the integration circuit 105, the maximum value (Gmax) of the G component is detected, and data representing the detected Gmax is sent to the D range calculation circuit 106. The D range calculation circuit 106 calculates the maximum luminance level of the subject from the information of the maximum value Gmax based on the photoelectric conversion characteristics of the sub-photosensitive pixels described in the figure, and calculates the maximum dynamic range necessary for recording the subject. .

  In this example, setting information indicating how much the reproduction dynamic range is to be set can be input through a predetermined user interface (described later). The D range selection information 107 designated by the user is sent from the CPU 56 to the D range calculation circuit 106. The D range calculation circuit 106 determines the dynamic range at the time of recording based on the dynamic range obtained by analyzing the imaging data and the D range selection information specified by the user.

  When the maximum dynamic range obtained from the imaging data is equal to or less than the D range of the D range selection information 107, the dynamic range obtained from the imaging data is adopted. When the maximum dynamic range obtained from the imaging data exceeds the D range of the D range selection information, the D range indicated by the D range selection information is employed.

  In accordance with the D range determined by the D range calculation circuit 106, the gamma coefficient of the gamma correction circuit 104 for low-sensitivity image data is controlled.

  The synchronization processing circuit 108 performs synchronization processing and YC conversion processing on the image data output from the gamma correction circuit 104. The luminance / color difference signal (YCrCr) generated by the synchronization processing circuit 108 is sent to various correction circuits 109, where correction processing such as edge enhancement and color difference matrix processing is performed. The low-sensitivity image data that has undergone the required correction processing in the various correction circuits 109 is compressed in the JPEG compression circuit 110 and recorded on the recording medium 52 as an image file different from the high-sensitivity image data.

  For high-sensitivity image data, an image design is made to the sRGB color standard, which is a typical characteristic of consumer displays. The photoelectric conversion characteristics for this sRGB color space are shown in FIG. By providing the imaging system with the conversion characteristics as shown in FIG. 11, it is possible to reproduce a preferable image with respect to luminance when reproducing an image using a normal display.

  On the other hand, in recent years, color reproduction design for an extended color space having a color space wider than sRGB may be performed in printing applications and the like.

  FIG. 12 shows an example of the sRGB space and the extended color space. In the figure, the inner side of the horseshoe shape indicated by reference numeral 120 indicates a color region that can be perceived by humans. The inside of the triangle indicated by reference numeral 121 indicates a color gamut that can be reproduced in the sRGB color space, and the inside of the triangle indicated by reference numeral 122 indicates a color gamut that can be reproduced in the extended color space. By changing the value of the linear matrix (the value of the matrix in the linear matrix circuits 92 and 102 described with reference to FIG. 10), the color region that can be reproduced can be changed.

  In the present embodiment, in applications intended for a color space other than sRGB, for example, printing applications, not only high-sensitivity image data but also low-sensitivity image data obtained by simultaneous exposure is used for image processing. The reproduction area and the luminance reproduction area are expanded to create a more preferable image. Different images corresponding to different dynamic ranges can be created by having different gammas depending on the reproduction range.

  FIG. 13 shows an encoding formula for the sRGB color gamut and an encoding formula for the extended color gamut. For example, as shown in the lower part of the figure (Case 2), a file corresponding to a reproducible luminance range can be created by making the encoding condition also correspond to a negative value or a value of 1 or more. For low-sensitivity image data, signal processing is performed according to the encoding condition corresponding to the expanded reproduction range, and a file is generated.

  Note that the depth of the bit is important because the highlight information has delicate information. Therefore, it is preferable to record data corresponding to sRGB in, for example, 8 bits, and to record data corresponding to the expanded reproduction area in, for example, 16 bits having a larger number of bits.

  FIG. 14 is a diagram illustrating an example of a directory (folder) structure of the recording medium 52. The camera 50 has a function of recording an image file in accordance with a DCF (Design rule for Camera File system: unified recording format of a digital camera defined by the Japan Electronic Industry Development Association [JEIDA]) standard.

  As shown in FIG. 14, a DCF image root directory having a directory name “DCIM” is formed immediately below the root directory, and at least one DCF directory exists immediately below the DCF image root directory. The DCF directory is a directory for storing image files that are DCF objects. A DCF directory name is defined by a 3-character directory number followed by 5 free characters (8 characters in total) according to the DCF standard. The DCF directory name may be automatically generated by the camera 50, or may be configured to be specified or changed by the user.

  The image file generated by the camera 50 is given a file name that is automatically generated according to a DCF naming rule or the like, and is stored in a designated or automatically selected DCF directory. A DCF file name that follows the DCF naming convention is defined by 4 free characters followed by a 4 character file number.

  Two image files respectively created from high-sensitivity image data and low-sensitivity image data acquired in the wide dynamic range recording mode are recorded in association with each other. For example, one file created from high-sensitivity image data (a file corresponding to a normal standard reproduction area, hereinafter referred to as a standard image file) is called “ABCD ****. JPG” according to the DCF naming convention. Naming (“****” is the file number) and the other file created from the low-sensitivity image data obtained by simultaneous imaging (file corresponding to the extended reproduction range, hereinafter referred to as the extended image file) For, add "b" to the end of the standard image file name (8 character strings excluding ".JPG") and name it "ABCD **** b.JPG". By storing the names in association with each other in this way, it is possible to select and use a file suitable for the characteristics at the time of output.

  As another example of the file name association method, a character such as “a” may be added to the end of the file name of the standard image file. The standard image file and the extended image file can be distinguished by changing the character string added after the file number. In addition, there is an aspect in which the free character part preceding the file number is changed. In addition, there is a mode in which the extension is changed between the standard image file and the extended image file. By sharing at least the file number, the relationship between the two files can be secured.

  The recording format of the extended image file is not limited to the JPEG format. As shown in FIG. 12, most of the colors are common in the sRGB color space and the extended color space. Therefore, when the captured image is encoded for the sRGB color space and the extended color space, and the difference between them is taken, most pixels of the image have a value of “0”. Therefore, for example, Huffman compression is performed on this difference, and one is used as the sRGB image file of the standard device and the other file is used as the difference image, so that the extended color space can be handled and the recording capacity can be reduced.

  FIG. 15 is a block diagram showing a mode in which low-sensitivity image data is recorded as a difference image as described above. In FIG. 15, the same or similar components as those in FIG. 10 are denoted by the same reference numerals, and the description thereof is omitted.

  An image generated from the high-sensitivity image data and an image generated from the low-sensitivity image data are sent to the difference processing circuit 132, and a difference image of these images is generated. The difference image generated by the difference processing circuit 132 is sent to the compression circuit 133, where it is compressed according to a predetermined compression method different from JPEG. The compressed image data file generated by the compression circuit 133 is recorded on the recording medium 52.

  FIG. 16 is a block diagram showing the configuration of the reproduction system. Information recorded on the recording medium 52 is read via the media interface unit 140. The media interface unit 140 is connected to the CPU 56 via a bus, and performs necessary signal conversion in order to transfer signals necessary for reading and writing of the recording medium 52 in accordance with instructions from the CPU 56.

  The compressed data of the standard image file read from the recording medium 52 is decompressed by the decompression processing unit 142, and decompressed in the high sensitivity image data restoration area 62C on the memory 62. The expanded high-sensitivity image data is sent to the display conversion circuit 146. The display conversion circuit 146 includes a reduction processing unit that converts the image size in accordance with the resolution of the display unit 54, a display signal generation unit that converts the display image generated by the reduction processing unit into a predetermined signal format for display, and including.

  The signal converted into a predetermined signal format for display in the display conversion circuit 146 is output to the display unit 54. Thus, the reproduced image is displayed on the display unit. Normally, only the standard image file is reproduced and displayed on the display unit.

  In addition, when creating an image with a wide reproduction range using the extended image file associated with the standard image file, the high-sensitivity image data of RGBG is restored from the data obtained by expanding the standard image file, This is stored in the high-sensitivity image data restoration area 62D on the memory 62.

  Further, the extended image file is read from the recording medium 52, decompressed by the decompression processing unit 148, restored to RGB low sensitivity image data, and stored in the low sensitivity image data restoration area 62 E on the memory 62. . The high-sensitivity image data and the low-sensitivity image data stored on the memory 62 are read out and sent to the synthesis processing unit (image adding unit) 150.

  The composition processing unit 150 multiplies the high-sensitivity image data by a coefficient, a multiplication unit that multiplies the low-sensitivity image data by a coefficient, and adds (synthesizes) the high-sensitivity image data and the low-sensitivity image data after the coefficient multiplication. ) Adding processing unit. Each coefficient (coefficient indicating the addition ratio) multiplied by the high sensitivity image data and the low sensitivity image data is variably set by the CPU 56.

  The signal generated by the synthesis processing unit 150 is sent to the gamma conversion unit 152. The gamma conversion unit 152 refers to the data in the ROM 60 under the control of the CPU 56 and converts the input / output characteristics so as to obtain a desired gamma characteristic. The CPU 56 performs control to switch the gamma characteristic in accordance with the reproduction range at the time of image output. The gamma-corrected image signal is sent to the YC conversion unit 153, and the RGB signal is converted into a luminance (Y) signal and a color difference signal (Cr, Cb).

  The luminance / color difference signal (YCrCr) generated by the YC conversion unit 153 is sent to various correction units 154. Various correction units 154 perform necessary correction processing such as edge enhancement (aperture correction) and color correction using a color difference matrix to generate a final image. The final image data generated in this way is sent to the display conversion circuit 146, converted into a display signal, and then output to the display unit 54.

  Although FIG. 16 illustrates an example in which an image is reproduced and displayed on the display unit 54 mounted on the camera 50, it is also possible to reproduce and display an image on an external image display device. In addition, by realizing a processing flow similar to that shown in FIG. 16 using a personal computer with a built-in image viewing application program, a dedicated image playback device, or a printer, it supports standard image reproduction and an expanded reproduction range. The reproduced image can be reproduced.

  FIG. 17 is a graph showing the relationship between the level of the final image (synthesized image data) obtained by synthesizing the high-sensitivity image data and the low-sensitivity image data and the relative subject luminance.

  Relative subject brightness represents subject brightness with reference to 100% of subject brightness that gives a level at which high-sensitivity image data is saturated. In FIG. 17, the image data is represented by 8 bits (0 to 255), but the number of bits is not limited to this.

  The dynamic range of the composite image is set through the user interface. In this example, it is assumed that the dynamic range can be set in six stages from D0 to D5. Since human senses are effective on a substantially log scale, the reproduction dynamic range is, for example, 100% −130% −170% in terms of relative subject luminance so that it becomes substantially linear when the log (logarithm) function is taken. It is configured so that it can be switched in stages such as 220% -300% -400%.

  Of course, the number of setting stages of the dynamic range is not limited to this example, it can be designed to an arbitrary number of stages, and continuous setting (no stage) is also possible.

  Depending on the setting of the dynamic range, the gamma coefficient of the gamma circuit, the synthesis parameter at the time of addition, the gain coefficient of the color difference signal matrix circuit, and the like are controlled. A non-volatile memory (ROM 60 or EEPROM 64) in the camera 50 stores table data defining various parameters, coefficients, etc. corresponding to the set dynamic range.

  18 and 19 show examples of user interfaces at the time of dynamic range selection operation. In the example shown in FIG. 18, an input box 160 for designating a dynamic range is displayed on the dynamic range setting screen transitioned from the menu screen. When a pull-down menu button 162 beside the input box 160 is selected using a predetermined operation means such as a cross button, a pull-down menu 164 showing a selectable dynamic range value (relative subject brightness) is displayed as shown in the figure.

  A dynamic range is set by selecting a desired dynamic range from the pull-down menu 164 using the cross button and pressing the OK button.

  In another example shown in FIG. 19, an input box 170 and a D range parameter axis 172 are displayed on the dynamic range setting screen. By using the operation means such as a cross button and moving the slider 174 along the D range parameter axis 172, the dynamic range can be arbitrarily specified in a range from 100% to a maximum of 400%. The setting value of the dynamic range shown in the input box 170 is changed according to the position of the slider 174. When the desired set value is obtained, the OK button is pressed according to the operation guide at the bottom of the screen to confirm (execute) the setting process. If the cancel button is pressed, the setting process is canceled and the original setting state is restored.

  18 and 19, the example of performing the dynamic range selection operation on the screen of the display unit 54 has been described. However, the dynamic range is selected by an operation member such as a dial type switch, a slide switch, or a push type switch. Embodiments are possible.

  In addition, since the required dynamic range differs depending on the shooting scene, there is also an aspect in which the dynamic range is automatically set by analyzing the captured image. Furthermore, a mode in which the setting of the dynamic range is automatically switched according to the photographing mode such as the portrait mode and the night view mode is also possible.

  The dynamic range information indicating how much information is recorded is recorded together with the image data in a file header or the like. The dynamic range information may be recorded in both the standard image file and the extended image file, or may be recorded in either one of the files.

  By adding dynamic range information in the image file, the information can be read by an image output device such as a printer, and the processing content such as composition processing, gamma conversion, and color correction can be changed to create an optimal image. It becomes possible.

  Even for print applications, images with soft gradations and reproduction of desirable skin colors are preferred for portraits. For example, for commercial photography and portrait or indoor photography. It is useful to create an extended image according to the application, such as for outdoor photography. In order to realize this, as described with reference to FIGS. 18 and 19, the camera 50 is provided with a user interface that can specify the luminance reproduction range of the extended image, and the user can select according to the usage and the shooting situation. It has become.

  Next, the operation of the camera 50 configured as described above will be described.

  20 to 22 are flowcharts showing the control procedure of the camera 50. When the camera power is turned on with the shooting mode selected, or when the playback mode is switched to the shooting mode, the control flow in FIG. 20 starts.

  When the shooting mode process starts (step S200), the CPU 56 first determines whether or not a mode for displaying a through image on the display unit 54 is selected (step S202). When the mode for turning on the display unit 54 at the start of the shooting mode (through image ON mode) is selected on the setup screen or the like, the process proceeds to step S204, where power is supplied to the imaging system including the CCD 20 and imaging is possible. become. At this time, the CCD 20 is driven at a constant imaging cycle in order to perform continuous imaging for live view display.

  The camera 50 of this example uses an NTSC video signal in the display unit 54, and the frame rate is set to 30 frames / second (1 field = 1/60 seconds to form one frame by two fields). ). In the case of the camera 50, since the same image is displayed in two fields, the image content is updated every 1/30 seconds. In order to update the image data of one screen at this period, the period of the vertical drive (VD) pulse of the CCD 20 is set to 1/30 second at the time of the through image. The CPU 56 gives a control signal for the CCD driving mode to the timing generator 82, and the timing generator 82 generates a CCD driving signal. Thus, continuous imaging by the CCD 20 is started, and a through image is displayed on the display unit 54 (step S206).

  While the through image is being displayed, the CPU 56 monitors the signal input from the shutter button and determines whether or not the S1 switch is turned on (step S208). If the S1 switch is in the OFF state, the process in step S208 loops and the through image display state is maintained.

  If the through image is set to OFF (non-display) in step S202, step S204 to step S206 are omitted and the process proceeds to step S208.

  Thereafter, when the photographer presses the shutter button and inputs a shooting preparation instruction (when the CPU 56 detects S1 = ON), the process proceeds to step S210, and AE and AF processes are performed. At this time, the CPU 56 changes the CCD drive mode to 1/60 seconds. The period of image capture from the CCD 20 is shortened, and AE / AF processing can be performed at high speed. The CCD driving period set here is not limited to 1/60 seconds, but can be set to an appropriate value such as 1/120 seconds. Shooting conditions are determined by AE processing, and focus adjustment is performed by AF processing.

  Thereafter, the CPU 56 determines signal input from the S2 switch of the shutter button (step S212). If the S2 switch is not turned on in step S212, it is determined whether or not S1 is released (step S214). If S1 is canceled in step S214, the process returns to step S208 to wait for input of a shooting instruction.

  On the other hand, if S1 is not canceled in step S214, the process returns to step S212 and waits for the input of S2 = ON. When the input of S2 = ON is detected in step S212, the process proceeds to step S216 shown in FIG. 21, and a photographing operation (CCD exposure) for acquiring a recording image is executed.

  Next, it is determined whether or not it is a mode for performing wide dynamic range recording, and the processing is controlled according to the setting mode. When the wide dynamic range recording mode is selected by a predetermined operation means such as a D range expansion mode switch, first, a signal is read from the main photosensitive pixel 21 (step S220), and the image data (main photosensitive portion) is read. Data) is written to the first image memory 62A (step S222).

  Next, a signal is read from the secondary photosensitive pixel 22 (step S224), and the image data (secondary photosensitive portion data) is written in the second image memory 62B (step S226).

  Thereafter, as described with reference to FIG. 10 or FIG. 15, required signal processing is performed on each of the main photosensitive portion data and the secondary photosensitive portion data (steps S228 and S230). The standard reproduction image file generated from the main photosensitive portion data and the extended reproduction image file generated from the secondary photosensitive portion data are associated with each other and recorded on the recording medium 52 (steps S232 and S234).

  On the other hand, in the normal recording mode in which the wide dynamic range recording is not performed in step S218, the signal is read from only the main photosensitive pixel 21 (step S240). The main photosensitive portion data is written in the first image memory 62A (step S242), and then the main photosensitive portion data is processed (step S248). Here, normal processing for creating an image from only the main photosensitive portion data is performed through the required signal processing described with reference to FIG. The image data generated in step S248 is recorded on the recording medium 52 according to a predetermined file format (step S252).

  When the image recording process is completed in step S234 or step S252, the process proceeds to step S256, and it is determined whether or not an operation for canceling the photographing mode has been performed. If a shooting mode release operation has been performed, the shooting mode is terminated (step S260). If the shooting mode release operation is not performed, the shooting mode state is maintained, and the process returns to step S202 in FIG.

  FIG. 22 is a flowchart of a subroutine regarding the processing procedure of the secondary photosensitive pixel data shown in step S230 of FIG. As shown in FIG. 22, when the processing of the secondary photosensitive pixel data starts (step S300), first, a setting process for dividing the screen into a plurality of integrated areas is performed (step S302), and G (green) for each area is performed. The average value of the components is calculated, and the maximum value (Gmax) of the G component is obtained (step S304).

  The luminance range of the subject is detected from the integrated information of the area thus obtained (step S306). On the other hand, dynamic range setting information (setting information on how much the dynamic range is expanded) set from a predetermined user interface is read (step S308). A final dynamic range is determined based on the luminance range of the subject detected in step S306 and the dynamic range setting information read in step S308 (step S310). For example, the dynamic range is automatically determined according to the luminance range of the subject, with the upper limit being the set D range indicated by the dynamic range setting information.

  Thereafter, the signal level of each color channel is adjusted by white balance processing (step S312). Also, various parameters such as a gamma correction coefficient and a color correction coefficient are determined based on the table data in accordance with the determined dynamic range (step S314).

  Gamma conversion and other processes are performed according to the determined parameters (step S316), and image data for extended reproduction is generated (step S318). After step S318, the process returns to the flowchart of FIG.

  Thus, when reproducing the image recorded on the recording medium 52, the reproduction range can be switched, and the standard reproduction image and the extended image can be switched and output as necessary. preferable. In this case, when reproducing the extended image, the gamma is adjusted so that the brightness of the main subject is substantially the same as that of the image for standard reproduction, and gradation is given to the high luminance part. Thus, the difference between the standard reproduction image and the extended image can be confirmed for the high luminance portion without changing the impression of the main subject portion.

  Further, when displaying the standard reproduction image on the display unit 54, it is determined whether or not the information for extended reproduction is recorded, and when the extended information is recorded (an associated file exists). As shown in FIG. 23, highlighting 180 is performed on the part corresponding to the difference between the two.

  For example, the difference between high-sensitivity image data and low-sensitivity image data is taken, and the part where the difference is a positive value (the part that includes the extended information that extends the reproduction range) is specially displayed (highlighted). )I do. The highlight display mode may be any display that can distinguish the target region from other regions, such as blinking of the corresponding part, enclosed display, brightness change, color change, or a combination thereof. The form is not particularly limited.

  In this way, by visualizing a region portion that can be reproduced in more detail using the associated extended information, the user can grasp the extensibility of image reproduction.

  In the above-described embodiment, a digital camera has been exemplified, but the scope of application of the present invention is not limited to this, and an electronic imaging function such as a video camera, a DVD camera, a mobile phone with a camera, a PDA with a camera, a mobile personal computer with a camera, etc. The present invention can also be applied to other imaging apparatuses provided.

  Further, the image reproduction means described with reference to FIG. 16 can be applied to an output device such as a printer or an image browsing device. That is, in place of the display conversion circuit 146 and the display unit 54 in FIG. 16, an image generation unit that generates an output image, such as a print image generation unit and a print unit, and finally the generated image is output. By providing the output unit, a good image utilizing the extended information can be obtained.

The top view which shows the structural example of the light-receiving surface of the CCD image pick-up element used for the electronic camera to which this invention is applied Sectional view along line 2-2 in FIG. Sectional drawing which follows the 3-3 line of FIG. 1 is a schematic plan view showing the overall configuration of the CCD shown in FIG. Plan view showing another structure example of CCD Sectional drawing which follows the 6-6 line of FIG. Plan view showing still another structural example of CCD Graph showing photoelectric conversion characteristics of main photosensitive pixel and secondary photosensitive pixel 1 is a block diagram showing a configuration of an electronic camera according to an embodiment of the present invention. The block diagram which shows the detailed structure of the signal processing part shown in FIG. Graph showing photoelectric conversion characteristics for sRGB color space The figure which showed the example of sRGB color space and extended color space Chart showing the encoding formula for the sRGB color gamut and the encoding formula for the extended color gamut The figure which shows an example of the directory (folder) structure of a recording medium The block diagram which shows the example of an aspect which records low sensitivity image data as a difference image Block diagram showing the configuration of the playback system A graph showing the relationship between the level of the final image (synthesized image data) obtained by combining high-sensitivity image data and low-sensitivity image data and the relative subject brightness The figure which shows the example of the user interface at the time of dynamic range selection operation The figure which shows the example of the user interface at the time of dynamic range selection operation Flow chart showing the control procedure of the camera of this example Flow chart showing the control procedure of the camera of this example Flow chart showing the control procedure of the camera of this example The figure which shows the example of a display of the image obtained by wide dynamic range imaging

Explanation of symbols

  DESCRIPTION OF SYMBOLS 20 ... CCD, 21 ... Photodiode area | region (main photosensitive pixel), 22 ... Photodiode area | region (secondary photosensitive pixel), 23 ... Vertical transfer path, 40 ... Color filter layer, 41 ... Micro lens, 52 ... Recording medium, 54 ... Display unit 56 ... CPU 62 ... Memory 97 ... JPEG compression circuit 105 ... Integration circuit 106 ... D range calculation circuit 110 ... JPEG compression circuit 132 ... Differential processing circuit 133 ... Compression circuit 180 ... Highlight display

Claims (9)

Recording means in which a first image information file composed of high-sensitivity image information formed by relatively high-sensitivity pixels is recorded;
Image display means for displaying output images,
When a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels and associated with the first image information file is recorded in the recording means, A first image created from one image information file is displayed on the image display means, and an image portion whose reproduction area is expanded by the second image information file with respect to the first image is displayed in the first image file . Display control means for highlighting on the display screen of one image;
An image processing apparatus comprising: Claim 1 Symbol mounting, characterized in Tei Rukoto recorded with high sensitivity image information respectively from the low-sensitivity image data different bit depths in said second image information file in said first image information file Image processing apparatus. The second image information file, an image processing apparatus according to claim 1 or 2, characterized in Tei Rukoto recorded is compressed by different compression method and the first image information file. A reading step of reading from the recording means a first image information file comprising high-sensitivity image information formed by relatively high-sensitivity pixels;
An image display step of outputting the images to the display device,
When a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels and associated with the first image information file is recorded in the recording means, A first image created from one image information file is displayed on the display device, and an image portion whose reproduction area is expanded by the second image information file with respect to the first image is displayed in the first image file . A display control process for highlighting on the image display screen,
An image processing method comprising: The reading step reads high-sensitivity image information in the first image information file and low-sensitivity image information in the second image information file recorded at different bit depths, respectively. The image processing method according to claim 4. 6. The image processing according to claim 4, wherein the reading step reads the first image information file and the second image information file that are compressed and recorded by different compression methods. Method. A reading function for reading out from the recording means a first image information file comprising high-sensitivity image information formed by relatively high-sensitivity pixels;
And an image display function of outputting the images to the display device,
When a second image information file composed of low-sensitivity image information formed by relatively low-sensitivity pixels and associated with the first image information file is recorded in the recording means, A first image created from one image information file is displayed on the display device, and an image portion whose reproduction area is expanded by the second image information file with respect to the first image is displayed in the first image file . Display control function to highlight on the image display screen,
An image processing program for causing a computer to realize the above. The reading function reads high-sensitivity image information in the first image information file and low-sensitivity image information in the second image information file recorded at different bit depths, respectively. An image processing program according to claim 7. 9. The image processing according to claim 7, wherein the reading function reads the first image information file and the second image information file that are compressed and recorded by different compression methods. program.

Images