JP2010193380A - Photographing apparatus, color conversion processing device, server information processor, client information processor, method of image data processing in server information processor, and method of color conversion processing in client information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce non-uniformity in image caused by an imaging device, an optical element, a shutter and the like without considerably increasing volume of multiband image data. <P>SOLUTION: An input profile information generating section 622 divides an image area into a plurality of divided areas and generates divided area representative characteristic information, that is information on a total spectral sensitivity characteristic representative of each divided area, based on a total spectral sensitivity characteristic corresponding to each of a plurality of pixels included in each of the divided areas. The input profile information generating section 622 generates input profile information by summarizing the divided area representative characteristic information together with other information. A data format processing section 624 adds the input profile information to multiband image data and records the multiband image data on a storage medium. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to processing of multiband image data with advanced color reproducibility, and in particular, an imaging device, a color conversion processing device capable of suppressing an increase in data size when transmitting and storing multiband image data, The present invention relates to a server information processing apparatus, a client information processing apparatus, an image data processing method in a server information processing apparatus, and a color conversion processing method in a client information processing apparatus.

  An ICC profile that is generally popular in the technical field that requires color management is related to a colorimetric value space (example: CIE1931XYZ) defined as a profile connection space (PCS) and a signal value of a device. It is defined as having data to attach. Here, ICC is an abbreviation for International Color Consortium.

  In the color management system, color reproduction is performed using the device profile data so that the colors in the PCS match between images of different devices. In recent years, a method has been proposed in which the PCS is extended from a conventional XYZ color space to a spectrum-based color space. For example, in a natural vision development project promoted by the National Institute of Information and Communications Technology, a spectral image system based on spectrum and multiple primary colors (sometimes called a spectral image system or a spectrum-based color reproduction system) is more faithful. A demonstration experiment was conducted to obtain a natural color reproduction with high degree.

  Non-Patent Document 1 discloses an example of a spectral image system. Briefly, in Non-Patent Document 1, it is assumed that not only CIE1931XYZ defined by the ICC profile but also spectral reflectance and spectral radiance (sometimes expressed as a reflection spectrum) can be used. Data required as a profile varies depending on what is used as the PCS. For example, consider a case where a color space based on spectral reflectance is used as a PCS. In this case, spectrum-based color reproduction processing is performed on image data output from an image input device such as a camera (hereinafter, image data output from an image input device such as a camera is referred to as original image data). Then, color conversion processing is performed, output device independent data is generated, and output to a display device or the like.

  In the color conversion process, spectral reflectance estimation is performed for each pixel, and output device-independent data is generated. This output device-independent data is spectral reflectance image data, and is device-independent color information (without device dependency). The output device independent data thus obtained is interpreted as a signal (data) that can be handled as a PCS. When performing the spectral reflectance estimation, a spectral reflectance estimation matrix is generally used. The spectral reflectance estimation matrix itself or information necessary to obtain it is included in the input profile, and the original image data in the color space used in the image input device is converted into image data in the PCS space.

  The image data in the PCS space obtained as described above can be stored and transmitted without being aware of the characteristics of the output destination device. However, when storage or transmission is considered, if each pixel constituting the image data holds spectral reflectance data, the data capacity increases. For example, if each of the pixels constituting the spectral reflectance image data has 401-dimensional data (data corresponding to a wavelength of 1 nm in the visible light wavelength band of 380 nm to 780 nm), the number of pixels is 401. In addition, it is necessary to store or transmit image data having a capacity multiplied by the bit depth.

  In this way, the capacity of the spectral reflectance image data is not efficient because it dramatically increases as compared with the capacity of the original image data. In order to avoid an increase in the volume of image data, the input profile information is added to the original image data, stored or transmitted, and the original image data is read or received by the device that receives the spectral reflectance image data in the PCS space. It is desirable to generate In this way, by adding or saving the input profile information to the original image data, it is possible to suppress an increase in the data capacity related to the storage and transmission.

Natural Vision (Next Generation Video Display / Transmission System) R & D Project [Movie] R & D Final Report, March 31, 2006, National Institute of Information and Communications Technology (Center Research Promotion Division))

  The input profile described above is uniformly applied to the data of each pixel constituting the original image data within one screen. However, an image input apparatus such as a camera may cause “unevenness” as described below. For example, when shooting a subject with uniform color and uniform brightness, such as a uniformly illuminated reference white plate, on the entire screen, the pixel values of all the pixels constituting the image data output from the image input device are Ideally it should be uniform.

  However, this is not always the case in reality, and the pixel values of the pixels constituting the output original image data vary. In this specification, what causes such variation is referred to as unevenness. This unevenness may be a factor that reduces the color reproducibility of the image. Examples of the unevenness include those described below.

1. Peripheral dimming characteristics of taking lens
This is caused by vignetting, cosine fourth law, and the like. In vignetting, among the luminous flux of subject light that passes through the photographic lens and reaches the imaging plane, the luminous flux that reaches the peripheral portion of the screen (the portion with a relatively high image height) is partially blocked by a lens frame or the like. This is a phenomenon in which the amount of light at the periphery of the screen is reduced. Cosine fourth law is the optical axis of the taking lens, the image plane illuminance by the light incident on the image plane at an angle alpha I.alpha, the image plane illuminance on the screen center upon the I _0, Iα = cos ⁴ This is a phenomenon of α × I ₀ . The peripheral dimming characteristics of the taking lens differ depending on the lens design. Also, even lenses with the same design vary depending on the set aperture value. When the photographing lens is a zoom lens, it varies depending on the set focal length. Strictly speaking, even with lenses of the same design, there are individual differences due to manufacturing errors, and there are also cases where the distance varies depending on the shooting distance (the focus adjustment position of the shooting lens, that is, the extension amount).

2. Light quantity reduction by obliquely entering the image sensor:
The decrease in the amount of light incident obliquely on the image sensor is due to the light incident obliquely on the image sensor surface because the photoelectric conversion part is located deeper than the image sensor surface (on-chip microlens formation surface). Then, only a part of the light transmitted through the on-chip microlens reaches the photoelectric conversion unit. This phenomenon varies depending on the configuration of the image sensor and the lens design. For example, by using a so-called telecentric configuration for the photographic lens, it is possible to suppress a decrease in the amount of light incident on the pixels present in the peripheral portion of the image area of the image sensor. Even with the same photographic lens, the light quantity reduction characteristic changes depending on the set aperture value. If the photographing lens is a zoom lens and the exit pupil position changes when the set focal length is changed, the light quantity reduction characteristic changes. In addition, regardless of whether the photographic lens is a zoom lens or a single focal length lens, the light quantity reduction characteristic also changes when the exit pupil position changes with focus adjustment.

3. Sensitivity unevenness of image sensor:
This is unevenness caused by the semiconductor manufacturing process. During the manufacturing process of the image sensor, for example, film formation, resist coating, exposure, development, etching, and addition of impurities on a silicon wafer having a size of 8 inches (about 200 mm), 12 inches (about 300 mm), 18 inches (about 450 mm) in diameter Such a semiconductor process is repeatedly performed. As a result, a plurality of image sensors are formed on the silicon wafer. These image pickup devices are diced, and the cut out individual chips are packaged to manufacture the image pickup device. Due to the characteristics of the semiconductor process, the distribution of the thickness of the film formed in the film formation process such as PVD or CVD is uneven due to the difference in the location in the silicon wafer, the thickness of the resist is uneven, When an impurity for forming the semiconductor layer is added, the amount added may be uneven.

  The unevenness as described above causes variations in photoelectric conversion efficiency among pixels constituting the image sensor. That is, the output signal value for the same incident light quantity varies. In addition, in a configuration having an amplifier for each pixel such as a CMOS image sensor, fluctuations in amplifier gain may also occur. For example, an image sensor having a relatively small image area such as a 1 / 2.5 type has a small chip size and is not easily affected by unevenness caused by the semiconductor process, but an image sensor having a relatively large image area. The chip size also increases, and there is a possibility that the difference in the level of the signal output from one end and the other end of the image area on the image sensor cannot be ignored. The distribution of the uneven sensitivity of the semiconductor element varies depending on the individual image sensor.

  In addition, the spectral transmission characteristics of the on-chip color filter may vary due to process variations when the on-chip color filter layer is formed by coating or photolithography.

4). Incident angle dependence of filter's spectral transmission characteristics:
There is a so-called multiband camera capable of performing color separation and imaging with a number of bands larger than 3, for example, 6 bands. As this multiband camera, a combination of an image sensor having three on-chip color filters of BGR (B = blue, G = green, R = red) and an interference filter having broadband and narrowband spectral transmission characteristics is used. Some perform color separation of bands.

  For example, a beam splitter that divides subject light transmitted through a photographic lens and guides it to two image sensors, and first and second image sensors having a BGR color separation optical system, and the first light of the beam splitter A first image sensor that receives subject light emitted from the emission unit, a second image sensor that receives subject light emitted from the second light emission of the beam splitter, a first light emission unit, and a first image sensor Between the first interference filter having a relatively broadband spectral transmission characteristic, and between the second light emitting unit and the second image sensor, and relatively What comprises a second interference filter having a narrow band spectral transmission characteristic is known.

  Each subject light divided by the beam splitter passes through a corresponding interference filter and enters a corresponding image sensor. The two image sensors may have the same specifications. Since the first interference filter has a spectral transmission characteristic of a relatively wide band and the second interference filter has a spectral transmission characteristic of a relatively narrow band, a 6-band image signal can be obtained from two image pickup devices. The interference filter used at this time can be obtained with an arbitrary spectral transmission characteristic by controlling the film thickness of the thin film formed on the transparent substrate. However, it has a characteristic that the spectral transmission characteristic is likely to change depending on the incident angle when light enters the interference filter. This is because if light is incident on the interference filter at an angle, it is equivalent to an increase in the thickness of the thin film. As a result, the spectral characteristics change between subject light that is transmitted through the same interference filter and is incident on the center of the screen and subject light that is incident on the periphery of the screen, so that the image signal also changes. This is the incident angle dependency of the filter spectral transmission characteristics.

  In order to cope with the unevenness caused by some of the above-mentioned factors, it is conceivable to apply a different input profile for each pixel instead of uniformly applying one input profile to the original image data. However, if an input profile to be applied for each pixel is attached to the original image data, the data capacity is greatly increased.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide a technique capable of reducing unevenness in an image without greatly increasing the capacity of multiband image data.

According to a first aspect of the present invention, there is provided an imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. An imaging device capable of generating multiband image data of a number of bands larger than the number of colors,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. A first spectral characteristic distribution information generation unit that generates characteristic distribution information;
A second spectral characteristic distribution information generation unit that generates second spectral characteristic distribution information that reflects non-uniformity of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. An input profile information generator to
A data format processing unit that performs format processing by adding the input profile information to the multiband image data;
The input profile information generation unit
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area A representative characteristic information generating unit that generates divided area representative characteristic information that is information;
An information aggregation processing unit that aggregates the divided region representative characteristic information together with other information to generate the input profile information solves the above-described problem.

According to a second aspect of the present invention, there is provided an imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. Multi-band image data generated by a photographing device capable of generating multi-band image data of a number of bands larger than the number of colors is accumulated, and the multi-band image data can be transmitted to the client information processing device A server information processing apparatus,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. A first spectral characteristic distribution information generation unit that generates characteristic distribution information;
A second spectral characteristic distribution information generation unit that generates second spectral characteristic distribution information that reflects non-uniformity of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. An input profile information generator to
A data format processing unit that performs format processing by adding the input profile information to the multiband image data;
The input profile information generation unit
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area A representative characteristic information generating unit that generates divided area representative characteristic information that is information;
An information aggregation processing unit that aggregates the divided region representative characteristic information together with other information to generate the input profile information.

According to a third aspect of the present invention, there is provided an imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. Multi-band image data generated by a photographing device capable of generating multi-band image data of a number of bands larger than the number of colors is accumulated, and the multi-band image data can be transmitted to the client information processing device An image data processing method in a server information processing apparatus,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. Generating characteristic distribution information;
Generating second spectral characteristic distribution information reflecting unevenness of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. To do
Adding the input profile information to the multiband image data and performing format processing;
Generating the input profile information;
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area Generating divided region representative characteristic information that is information;
The divided area representative characteristic information is aggregated together with other information to generate the input profile information.

  According to the present invention, the entire image is divided into a plurality of divided areas, and the divided areas representing each divided area based on the spectral sensitivity characteristics corresponding to each of the plurality of pixels included in each divided area. Representative characteristic information is obtained, and input profile information generated by aggregating this divided area representative characteristic information together with other information is added to the multiband image data, and color reproduction is performed using this input profile information. It is possible to effectively reduce the capacity of the input profile information while suppressing image unevenness.

FIG. 3 is a diagram conceptually illustrating a state in which original image data generated by a photographing device is transferred to an information processing device in the first embodiment of the present invention. FIG. 2 is a block diagram schematically illustrating an internal configuration of a photographing apparatus and a photographing lens. It is a figure which shows an example of the combination of the spectral transmission characteristic of a filter, and the spectral sensitivity characteristic of an image pick-up element. It is a figure which shows another example of the combination of the spectral transmission characteristic of a filter, and the spectral sensitivity characteristic of an image pick-up element. It is a figure which shows an example of the spectral sensitivity characteristic obtained by the combination of the spectral transmission characteristic of a filter, and the spectral sensitivity characteristic of an image pick-up element. It is a block diagram which illustrates roughly the internal structure of an incidental information addition process part. It is a block diagram which illustrates roughly the internal structure of an input profile information generation part. It is a figure explaining the structural example of input profile information. It is a figure which illustrates conceptually the information of a spectral sensitivity pattern. It is a figure explaining a division area. It is a figure which illustrates notionally that a plurality of sets of information including divided area representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each area are generated. It is a schematic flowchart explaining the production | generation process procedure of the division area representative characteristic information, representative characteristic designation | designated information, and spectral sensitivity correction coefficient for every area | region performed with an imaging device. It is a schematic flowchart explaining the input profile information generation process procedure performed with an imaging device. FIG. 3 is a block diagram schematically illustrating an internal configuration of a color conversion processing unit that performs color conversion processing on original image data transferred from a photographing apparatus. It is a schematic flowchart explaining the color conversion process procedure performed with an information processing apparatus. It is a schematic flowchart explaining the process sequence performed with an information processing apparatus following the process procedure shown by the flowchart of FIG. 10A. In the second embodiment of the present invention, original image data generated by a photographing apparatus and attached with primary input profile information is transferred to and stored in the first information processing apparatus, and the original image data is stored in the first image data. It is a figure which illustrates notionally a mode that it processes with an information processing apparatus, and is transferred to a 2nd information processing apparatus. It is a block diagram which illustrates roughly the internal structure of an incidental information addition process part. It is a figure explaining the structural example of primary input profile information. It is a block diagram which illustrates roughly the internal structure of an incidental information addition process part. It is another block diagram which illustrates roughly the internal structure of an incidental information addition process part. It is a schematic flowchart explaining the procedure of the multiband image data acquisition process performed by the 2nd information processing apparatus (client). It is a schematic flowchart explaining the procedure of the multiband image data transmission process performed by the 1st information processing apparatus (content server).

− First embodiment −
FIG. 1 is a diagram illustrating a photographing apparatus 100 to which the present invention is applied, and an information processing apparatus 140 that performs color conversion processing on original image data generated by the photographing apparatus 100 and performs printing, display, and the like. FIG. 2 is a block diagram schematically showing the internal configuration of the photographing apparatus 100 and the photographing lens 110. In the present embodiment, it is assumed that imaging device 100 is a multiband imaging device configured to be able to generate 6-band original image data, and the imaging lens can be replaced. Further, it is assumed that the photographing lens 110 is a zoom type.

  The photographing apparatus 100 has a camera CPU 102, and the photographing lens 110 has a lens CPU 112. When the photographing lens 110 is attached to the photographing apparatus 100 and the photographing apparatus 100 is turned on, the camera CPU 102 and the lens CPU 112 communicate with each other. Various information related to the photographing lens 110, which will be described in detail later, is output from the lens CPU 112 to the camera CPU 102. The information includes information related to the peripheral dimming characteristics of the taking lens 110. Further, when the user performs an operation of changing the set focal length of the photographing lens 110, information related to the set focal length is output from the lens CPU 112 to the camera CPU 102.

  The setting of the aperture of the taking lens 110 is controlled by the camera CPU 102. For example, a case where the photographing mode of the photographing apparatus 100 is set to the program exposure mode will be described. The camera CPU 102 determines the shutter speed and the aperture value from the photometric result and the sensitivity (equivalent ISO sensitivity) of the image sensor preset in the photographing apparatus 100. The camera CPU 102 issues an aperture control signal to the lens CPU 112 based on the determined aperture value. The lens CPU 112 controls the aperture value of the taking lens 110. That is, the aperture value set at the time of shooting can be grasped by the camera CPU 102. In this specification, the photographing apparatus 100 is described as a single-lens reflex digital still camera in which the photographing lens 110 can be replaced. However, a movie capable of capturing a moving image even if the photographing lens is fixed. It may be a camera. Alternatively, it may be a mobile phone or a personal digital assistant (PDA) incorporating a photographing device.

  The information processing apparatus 140 may be configured by a personal computer (hereinafter referred to as a PC) or the like, and can process and store at least one of image data, storage, display, printing, and the like. In addition, a dedicated device may be used. In the following, it is assumed that the information processing apparatus 140 includes a PC 142, a display device 144, a keyboard 146, and the like. A printer 150 or the like can be connected to the information processing apparatus 140. The PC 142 can be configured to be connectable to another PC, server, or the like via a local area network (LAN), the Internet, or the like. In this specification, the display device 144 and the printer 150 are collectively referred to as an image output device.

  Image data can be transferred from the photographing apparatus 100 to the information processing apparatus 140 via wired communication using a USB cable, LAN cable, IEEE 1394 cable, HDMI cable, wireless communication using light or electromagnetic waves, a memory card, etc. Configured.

  Referring to FIG. 2, the imaging apparatus 100 includes two imaging elements 218 and 220, a beam splitter 230 that divides subject light transmitted through the imaging lens 110 and guides it to the two imaging elements 218 and 220, and a beam splitter 230. Shutters 210 and 212 and optical filters (hereinafter simply referred to as “filters”) 214 and 216 disposed between the imaging elements 218 and 220, respectively.

  The imaging apparatus 100 also includes a first buffer 202 and a second buffer 204 that temporarily store image data output from the image sensors 218 and 220, and image data stored in the first buffer 202 and the second buffer 204. An image processing unit 206 that performs processing to generate multiband image data, and a driver 208 that drives shutters 210 and 212 and the like based on a shutter control signal output from the camera CPU 102 are included. As the shutters 210 and 212, a focal plane shutter or the like can be used. Instead of these focal plane shutters, a lens shutter can be incorporated in the photographing lens 110.

  The photographing apparatus 100 further includes a memory 205. The memory 205 includes a RAM having an excellent access speed and a flash memory that does not require power for storage retention. The first buffer 202 and the second buffer 204 described above may be included in the memory 205. In this case, areas of the first buffer 202 and the second buffer 204 are secured in the RAM of the memory 205. The flash memory stores firmware executed by the camera CPU 102, information unique to each photographing apparatus 100, operation status information of the photographing apparatus 100, and the like.

  In the present embodiment, each of the image sensors 218 and 220 will be described as a single plate type having a Bayer array BGR on-chip color filter. However, the present invention is not limited to this example. For example, it may have a configuration in which color separation optical systems each including a dichroic prism or the like are disposed at positions facing the light emitting portion of the beam splitter 230. In this case, the subject light divided into two by the beam splitter 230 and transmitted through the filters 214 and 216 is color-separated by the color separation optical system. Each color-separated subject light is emitted to a plurality of single-color image sensors arranged at a position where the color-separated subject light of each color is received.

  The filters 214 and 216 have an interference filter function. When optical elements such as an optical low-pass filter (OLPF) are disposed between the light emitting portion of the beam splitter 230 and each of the imaging elements 218 and 220, a thin film layer constituting an interference filter on the surface of those optical elements Can also be formed. Note that the filters 214 and 216 do not necessarily have to function as interference filters. That is, only one of the filters 214 and 216 has the function of an interference filter, and the other may be another optical element, for example, a simple plane-parallel glass or an OLPF. Furthermore, the filters 214 and 216 are not necessarily limited to filters using light interference, and the present invention can be used in the same manner as long as the characteristics change when light enters obliquely. . In the following description, it is assumed that both the filters 214 and 216 have an interference filter function, the filter 214 has a relatively broad spectral transmission characteristic, and the filter 216 has a relatively narrow spectral transmission characteristic. To do. The spectral transmission characteristics of these filters 214 and 216 will be described later.

  The photographing apparatus 100 further includes an accompanying information addition processing unit 600. The supplementary information addition processing unit 600 generates input profile information to be described later when the photographing apparatus 100 performs photographing. The incidental information addition processing unit 600 adds these input profile information to the multiband image data generated by the image processing unit 206 and records it as original image data in the storage medium 240. As the storage medium 240, a so-called removable medium such as a memory card or a small hard disk that can be detachably attached to the photographing apparatus 100 can be used. In the following description, it is assumed that the supplementary information addition processing unit 600 is implemented when the camera CPU 102 executes the firmware stored in the memory 105.

  The taking lens 110 includes a lens CPU 112, an aperture driver 114, an aperture unit 116, a lens driving unit 118, and a memory 120. The aperture unit 116 includes a plurality of aperture blades that form a bright aperture, and an actuator that drives the aperture blades to open and close. An electromagnetic actuator, for example, a stepping motor, can be used as the actuator that opens and closes the diaphragm blades. The lens driving unit 118 includes an actuator for moving the focusing lens in the photographing lens 110 along the optical axis direction of the photographing lens 110, such as an ultrasonic motor or an electromagnetic drive motor.

  The lens driving unit 118 is also configured to be able to output information corresponding to the position of the focusing lens to the lens CPU 112. As an example, the lens driving unit 118 constantly monitors a signal output from an encoder capable of generating a signal according to the amount of movement of the focusing lens from the infinity shooting position, and sends information based on the latest lens position to the lens CPU 112. It is configured to output at any time. Further, even when the photographing lens 110 is configured so that manual focus adjustment by the user of the photographing apparatus 100 is possible, the lens driving unit 118 is configured to output information according to the latest focusing lens position to the lens CPU 112 as needed. Is possible. Hereinafter, information corresponding to the position of the focusing lens is simply referred to as “lens position information”.

  3 and 4 show examples of spectral transmission characteristics of the filters 214 and 216. FIG. In the graphs shown in FIGS. 3 and 4, the horizontal axis indicates the wavelength, and the vertical axis indicates the transmittance. These graphs also show the relative spectral sensitivity of the pixels having the B, G, and R wavelength bands in the image sensors 218 and 220 as well as the spectral transmission characteristics of the filters 214 and 216. As shown in FIG. 3, the filter 214 has a spectral transmission characteristic with a relatively wide band.

  On the other hand, the filter 216 has a relatively narrow band spectral transmission characteristic as shown in FIG. Hereinafter, the spectral transmission characteristics of the filters 214 and 216 are simply expressed as “having broadband spectral transmission characteristics” and “having narrow band spectral transmission characteristics”. As shown in FIG. 3, the filter 214 has a characteristic that the transmittance sharply decreases on the long wavelength side in the visible range. It can be referred to as having a broadband spectral transmission characteristic in that light in the visible range can be transmitted almost uniformly. As shown in FIG. 4, the filter 216 has a falling part (part where the spectral transmittance sharply decreases, two places) and a rising part (part where the spectral transmittance rises rapidly, one place). ) Is determined so as to divide the spectral sensitivity profile (mountain profile) of the pixels having the B, G, and R wavelength bands in the image sensor 220 in the vicinity of the peak. It can be referred to as having a narrow-band spectral transmission characteristic in that the wavelength band of light that can be transmitted by the filter 216 is narrower than that of the filter 214.

  The filter 214 having a broadband spectral transmission characteristic can be omitted as described above. For example, in FIG. 2, a configuration in which filters 214 and 216 in which a thin film layer is formed on a transparent substrate is interposed between the exit surface of the beam splitter 230 and the imaging elements 218 and 220, respectively. When the filter 214 is omitted, it is desirable to use a transparent transparent substrate instead to equalize the equivalent optical path length to the two image sensors 218 and 220 and to equalize the aberration characteristics of the photographing lens 110. . In addition, an OLPF, an infrared cut filter, or the like is interposed between the exit surface of the beam splitter 230 and the imaging elements 218 and 220, and a thin film layer is formed on the surface to form the filters 214 and 216. In this case, the thin film layer corresponding to the filter 214 may be omitted.

  FIG. 5 illustrates an image sensor 218 that is determined by a combination of the spectral transmission characteristics of the filters 214 and 216, the spectral transmission characteristics of the B, G, and R on-chip color filters, and the spectral sensitivity characteristics of the pixels that form the imaging element. , 220 is a diagram showing the spectral sensitivities of the pixels of each of the B, G, and R colors. That is, band 1, band 2, and band 3 indicate the spectral sensitivity characteristics of each pixel provided with the B on-chip color filter, the G on-chip color filter, and the R on-chip color filter of the image sensor 218, respectively. Band 4, band 5, and band 6 indicate spectral sensitivity characteristics of each pixel provided with the B on-chip color filter, the G on-chip color filter, and the R on-chip color filter of the image sensor 220, respectively.

  FIG. 6A is a block diagram schematically showing the configuration of the supplementary information addition processing unit 600. The supplementary information addition processing unit 600 performs processing for adding the input profile information to the multiband image data generated by the image processing unit 206 to generate original image data. The original image data generated by the supplementary information addition processing unit 600 is recorded in the storage medium 240.

  Before describing the incidental information addition processing unit 600, the sensor information storage unit 630 and the filter information storage unit 632 will be described. These sensor information storage unit 630 and filter information storage unit 632 are provided in a flash memory in the memory 205.

  Sensor information is stored in the sensor information storage unit 630. The sensor information includes information related to bias / tone characteristics of the image sensors 218 and 220 and information related to unevenness in spectral sensitivity (hereinafter simply referred to as “sensitivity unevenness information”). As described in detail below, the sensitivity unevenness information is obtained due to at least one of the image sensors 218 and 220, the processing system that processes the image signals output from the image sensors 218 and 220, and the shutter. This is information reflecting the unevenness of spectral sensitivity that occurs equivalently on the image area of the element.

  Sensitivity unevenness caused by the image sensors 218 and 220 is spectral sensitivity unevenness caused by process variations such as film formation and etching in the process of manufacturing the image sensors 218 and 220 as described in the related art.

  Sensitivity unevenness due to a processing system that processes image signals output from the image sensors 218 and 220 is uneven due to variations in characteristics of, for example, A / D converters and amplifiers formed in the image sensors 218 and 220. . For example, in the case where an amplifier is provided for each pixel or column in the image sensors 218 and 220, if the gains of these amplifiers vary, it may cause uneven spectral sensitivity. In addition, in the case where a plurality of A / D converters are provided in the image sensors 218 and 220 so that image signals can be read simultaneously from a plurality of channels, unevenness in spectral sensitivity occurs due to variations in the characteristics of the A / D converters. sell. As a result, the image data obtained by processing the image signals output from the image sensors 218 and 220 may be included in the image data even if uniform light is incident on the image areas of the image sensors 218 and 220. Each pixel value varies. Due to this variation in pixel values, color unevenness occurs.

  When the exposure accuracy of the shutter varies, the exposure amount of the image surface varies, and as a result, the image data varies. For example, if there is a difference in the exposure amounts obtained by the shutters 210 and 212, the spectral characteristics of the image data generated by processing the image signals obtained from the image sensors 218 and 220 will be shifted.

  The variation in the pixel value of the image data affected by the unevenness due to the above factors can be understood as if the unevenness of the spectral sensitivity of the image sensors 218 and 220 is the cause. That is, even if the cause of unevenness in the images obtained from the image sensors 218 and 220 is the unevenness of the spectral sensitivity of the image sensors 218 and 220, the processing system for processing the image signal, or the shutter, they are all image data. This is reflected as a variation in the pixel value. Therefore, the sensor information storage unit 630 stores information on the spectral sensitivity unevenness caused by the shutters 210 and 212 and the processing system that processes the image signals output from the image sensors 218 and 220 and the image sensors 218 and 220. , 220 are stored as representing the nonuniformity in spectral sensitivity that occurs equivalently on the 220 image area.

  The unevenness in spectral sensitivity due to the image pickup devices 218 and 220 and the processing system that processes the image signals output from the image pickup devices 218 and 220 can be obtained, for example, by performing calibration during the manufacturing process of the photographing apparatus 100. As an example of calibration, a translucent milky white cap is attached to the lens mount portion of the photographing apparatus 100, and image data obtained by photographing the photographing apparatus 100 toward a surface light source having a known spectral radiance is processed. Is possible. From variations in pixel values of the obtained image data, variations in spectral sensitivity corresponding to the pixels of the image sensors 218 and 220 are obtained. From this variation in spectral sensitivity, it is possible to generate information related to the spectral sensitivity distribution on the light receiving surfaces of the image sensors 218 and 220. When performing this calibration, a relatively long shutter speed is effective in reducing the influence of the shutter accuracy on the calibration result.

  The nonuniformity in spectral sensitivity due to the shutters 210 and 212 can be measured by performing calibration in the manufacturing process of the photographing apparatus 100. For example, a photographing lens for calibration is attached to the photographing apparatus 100 and photographing is performed so that a uniform luminance surface is captured on the entire screen. At this time, images are taken at a shutter time (for example, 1/32 seconds) at which exposure unevenness is unlikely to occur, and at a shutter time (for example, 1/8192 seconds) at which exposure unevenness is likely to occur, and an image obtained from the image sensor at each shutter time. Compare signals. Then, by comparing pixel values for each pixel, it is possible to estimate unevenness caused by the shutter.

  The example of performing calibration in the manufacturing process of the photographing apparatus 100 has been described above, but calibration can also be performed in the use process of the photographing apparatus 100 as described below. Assuming that the shutters 210 and 212 are focal plane shutters, sensors for detecting the traveling timing of the front curtain and rear curtain are incorporated in the shutter. That is, a sensor such as a photo interrupter or a photo reflector is incorporated in the shutters 210 and 212. With these sensors, it is possible to detect the time from when the front curtain passes the vicinity of the sensor to when the rear curtain passes the vicinity of the same sensor when the shutter is operated. From the difference between the detected time and the target shutter speed, the spectral sensitivity unevenness caused by the shutters 210 and 212 can be obtained. That is, if the exposure accuracy differs between the shutters 210 and 212, an error occurs in the relative value between the image signals output from the image sensors 218 and 220, and the color reproducibility of the resulting image data is affected. Will be affected. In the present specification, this is also treated as unevenness.

  As a characteristic of the focal plane shutter, the width of the slit formed by the front curtain and the rear curtain may vary during the exposure operation. This also causes unevenness. For example, three sensors for detecting the timing of the leading and trailing curtains are incorporated in the shutter, and the time intervals when the leading and trailing curtains travel at the curtain starting point, middle point, and driving completion point are measured. By doing so, it is possible to detect unevenness due to fluctuations in the slit width. Unevenness detected in this way can also be handled as unevenness in spectral sensitivity of the image sensors 218 and 220.

  The sensor information storage unit 630 stores sensitivity unevenness information and information on bias / tone characteristics obtained as described above as sensor information. Note that the sensitivity unevenness information stored in the sensor information storage unit 630 is not necessarily stored in all of the image sensors 218 and 220, the processing system that processes the image signals output from the image sensors 218 and 220, and the shutter. It is not necessary to store the information that is caused by at least one of them.

  Information related to the spectral transmission characteristics of the filters 214 and 216 is stored in the filter information storage unit 632. The filter information storage unit 632 also includes information related to changes in spectral transmission characteristics when light is incident on the filters 214 and 216 obliquely, that is, information that can derive changes in spectral transmission characteristics with respect to changes in the incident angle. Can also be memorized.

  The supplementary information addition processing unit 600 will be described. The incidental information addition processing unit 600 includes a lens information acquisition unit 602, a shooting information acquisition unit 604, a subject characteristic information input unit 606, a shooting illumination light information input unit 608, and a peripheral light reduction derivation unit 610. The incidental information addition processing unit 600 also includes an incident angle deriving unit 612, a sensitivity unevenness deriving unit 614, a bias / tone characteristic extracting unit 616, an input profile information generating unit 622, and a data format processing unit 624.

  The lens information acquisition unit 602 acquires lens information of the photographing lens 110 attached to the photographing apparatus 100 from the lens CPU 112. This lens information includes, for example, the focal length of the photographing lens (focal length range from the wide-angle end to the telephoto end in the case of a zoom lens), the open F value, the peripheral dimming characteristics, the exit pupil position, and the like. it can. The lens information may also include information related to spectral transmission characteristics and information that can identify the individual photographing lens 110, such as information that can specify the manufacturer, model, serial number, and the like of the photographing lens 110.

  Information on the exit pupil position includes the distance along the optical axis direction from the imaging plane to the exit pupil position when the photographic lens is focused so that an object at infinity is in focus, or the photographic lens 110 (imaging device). 100) information related to the distance along the optical axis direction from the mount surface to the exit pupil position. The lens information acquisition unit 602 outputs the information acquired as described above to the peripheral dimming deriving unit 610 and the incident angle deriving unit 612.

  The shooting information acquisition unit 604 acquires various types of shooting information when a shooting operation is executed by the shooting apparatus 100. For example, the shooting information includes the shooting date and time, the sensitivity set by the image sensors 218 and 220 (equivalent ISO sensitivity), and the focal length (hereinafter referred to as “photographing distance”) set when the shooting lens 110 is a zoom lens. (Referred to as a set focal length), shutter speed, set aperture value, exposure correction amount, shooting distance (information for obtaining the exit pupil position at the time of shooting), and the like. The imaging information acquisition unit 604 outputs the information acquired as described above to the peripheral dimming deriving unit 610, the incident angle deriving unit 612, and the input profile information generating unit 622.

  The subject characteristic information input unit 606 inputs information used for estimating the spectral reflectance of the subject in the color conversion processing described later. For example, there is a method called Wiener estimation as an example of a method for estimating the spectral reflectance of a subject. When Wiener estimation is used for estimating the spectral reflectance of the subject, the subject characteristic information input unit 606 inputs statistical information of the subject.

  For example, the user can call the menu screen of the photographing apparatus 100 and perform settings for specifying the type of subject such as human skin, natural scenery, buildings, and the like. Based on the setting result by the user, the subject characteristic information input unit 606 reads out statistical information effective for estimating the spectral reflectance from the memory 205. Then, the subject characteristic information input unit 606 outputs this statistical information to the input profile information generation unit 622.

  Note that a Markov model can be applied as a probability model when estimating the spectral reflectance of a subject by Wiener estimation. In that case, the subject characteristic information input unit 606 can be omitted. The subject characteristic information input unit 606 outputs the information input as described above to the input profile information generation unit 622.

  The photographing illumination light information input unit 608 inputs information used for estimating the spectral reflectance of the subject in the color conversion process described later. For example, when the photographing apparatus 100 has a spectrophotometric apparatus, the photographing illumination light information input unit 608 can input from the spectrophotometric apparatus spectral characteristics of photographing illumination light, that is, light illuminating a subject at the time of photographing. it can.

  As the configuration of the spectrophotometer, various configurations already known can be used. For example, the imaging illumination light incident from the light receiving part is guided to one end of the optical fiber, the light emitted from the other end is collimated and then dispersed using a diffraction grating, and the position (distribution) and intensity of each wavelength of the dispersed light Can be measured using a line sensor or the like to measure the spectral characteristics of the photographic illumination light.

  Alternatively, the imaging illumination light information input unit 608 can input the result of measuring the intensity of light of each color of B, G, and R with a sensor without using the above-described configuration, but the accuracy is determined using a spectrophotometer. Reduced compared to things. The photographing illumination light information input unit 608 outputs the photographing illumination light information input as described above to the input profile information generation unit 622.

  The above-described photographing illumination light information will be further described. For example, if the subject is illuminated with a single illumination light, such as a mountain surface illuminated by the setting sun or a still life placed under the illumination of a fluorescent lamp, there is only one type of shooting illumination light information. This photographic illumination information is uniformly applied in the color conversion processing described later.

  On the other hand, in situations where the background is illuminated with a fluorescent light and the main subject is illuminated with a tungsten light spotlight, different illumination light information is applied to each of multiple areas obtained by dividing the scene. It is also possible to do. In this case, at the time of shooting, a spectroscopic device having a relatively narrow measurement range (field range) is used to scan with a standard reflector (reference white plate) placed in the shooting scene, and the shooting illumination light within the field of view is scanned. The distribution can also be measured in advance. In this case, the imaging illumination light information input unit 608 inputs information related to the spectral characteristics of the imaging illumination light measured as described above, and outputs the information to the input profile information generation unit 622.

  The peripheral dimming deriving unit 610 inputs information related to the peripheral dimming characteristics of the taking lens 110 from the lens information acquisition unit 602. The peripheral dimming deriving unit 610 also inputs information related to the set focal length, aperture value, and imaging distance (or information that can determine the exit pupil position at the time of imaging) from the imaging information acquisition unit 604. Based on the information input as described above, the peripheral light reduction deriving unit 610 derives the amount of decrease in image plane illuminance (image plane illuminance distribution) on the light receiving surfaces of the image sensors 218 and 220, and generates input profile information. Output to the unit 622.

  The incident angle deriving unit 612 obtains information related to the exit pupil position from the lens information acquisition unit 602, and sets focal length and imaging distance (or information capable of obtaining the exit pupil position at the time of imaging) from the imaging information acquisition unit 604. Enter information related to. Based on such information, the incident angle deriving unit 612 derives an incident angle (hereinafter referred to as a filter incident angle) when subject light is incident on the filters 214 and 216, and inputs information related to the incident angle to the input profile. The information is output to the information generation unit 622.

  Sensitivity unevenness deriving unit 614 receives sensor information from sensor information storage unit 630 and derives sensitivity unevenness, that is, spectral sensitivity distributions on the light receiving surfaces of imaging elements 218 and 220, respectively. The sensitivity unevenness derivation unit 614 outputs information related to the spectral sensitivity of each pixel including the sensitivity unevenness to the input profile information generation unit 622.

  The bias / tone characteristic extraction unit 616 extracts the bias / tone characteristics of the image sensors 218 and 220 from the information recorded in the sensor information storage unit 630 and outputs them to the input profile information generation unit 622.

  FIG. 6B is a block diagram schematically showing the internal configuration of the input profile information generation unit 622. The input profile information generation unit 622 includes an area division processing unit 620, a filter characteristic change amount derivation unit 618, an overall spectral sensitivity characteristic derivation unit 626, a representative characteristic information generation unit 628A, a correction coefficient generation unit 628B, and information aggregation. And a processing unit 628C.

  The input profile information generation unit 622 includes a photographing information acquisition unit 604, a photographing illumination light information input unit 608, a subject characteristic information input unit 606, a bias / tone characteristic extraction unit 616, an incident angle derivation unit 612, a filter information storage unit 632, a sensitivity. Based on the information output from the unevenness deriving unit 614 and the peripheral dimming deriving unit 610, input profile information to be described in detail later with reference to FIG. 7A is generated, and this information is output to the data format processing unit 624. .

  The filter characteristic change amount deriving unit 618 reads information related to the spectral transmission characteristics of the filters 214 and 216 from the filter information storage unit 632. Then, the filter characteristic change amount deriving unit 618 calculates a filter characteristic shift amount corresponding to the filter incident angle from the information related to the spectral transmission characteristic and the information related to the filter incident angle output from the incident angle deriving unit 612. Information relating to the filter characteristic shift amount is output to the total spectral sensitivity characteristic deriving unit 626.

  Processing in the filter characteristic change amount deriving unit 618 will be described. Consider a subject image that is emitted from the exit pupil position of the photographic lens 110, passes through the filters 214 and 216, and forms an image on the light receiving surfaces of the image sensors 218 and 220. The center of the image area of the image sensors 218 and 220 (the center of the screen), that is, the subject light incident on the position where the image height is zero (for the sake of simplicity, only the principal ray is considered here) Incident light is incident so as to be orthogonal to the incident surface (filter incident angle = 0 degree).

  On the other hand, subject light incident on a position away from the center of the screen (position having a certain image height) is incident on the incident surfaces of the filters 214 and 216 obliquely. The higher the image height, the larger the incident angle of the filter. Therefore, the thin film when subject light passes through the filters 214 and 216 is equivalently thick. Therefore, the spectral transmission characteristics of the filters 214 and 216 change according to the image height of the subject image formed on the light receiving surfaces of the image sensors 218 and 220.

  Since the arrangement pitch of the pixels on the image sensors 218 and 220 is known, the image height at the position where each pixel is located is obtained by calculation on the basis of the pixel located at the center of the image area (image height = 0). Can do. Therefore, from the exit pupil position and the image height, the incident angle deriving unit 612 can determine the incident angle of the filter when the light ray incident on each pixel passes through the filters 214 and 216. The filter characteristic change amount deriving unit 618 obtains the spectral transmission characteristics of the filters 214 and 216 corresponding to the respective pixels in the image area from the filter incident angle obtained by the incident angle deriving unit 612, and derives the total spectral sensitivity characteristic. Output to the unit 626.

  The region division processing unit 620 performs region division processing based on the filter incident angle derived by the incident angle deriving unit 612. One example will be described. For example, in a horizontally long rectangular screen having an aspect ratio of 4: 3, the filter incident angle corresponding to the most peripheral position of the screen (hereinafter referred to as the maximum incident angle) is 30 degrees in the horizontal direction of the screen and the vertical direction of the screen. 22.5 degrees (all shall be half-width). The area division processing unit 620 divides these maximum incident angles by 5, for example. As a result, a value of 6 in the horizontal direction and 4.5 in the vertical direction is obtained. Since the maximum incident angle is a half angle, the region division processing unit 620 doubles each of the calculation results, the number of regions in the horizontal direction of the screen (the number of regions in the X direction) is 12, and the number of regions in the vertical direction of the screen (the region in the Y direction) The number is determined to be 9. The area division processing unit 620 outputs information on the number of X-direction areas and the number of Y-direction areas to the representative characteristic information generation unit 628A.

  As described above, the spectral transmission characteristics of the filter gradually change according to the change in image height. Accordingly, when the image area is divided and the filter spectral transmission characteristics change from one end to the other end in each of the divided areas (hereinafter referred to as “divided areas”), the size of the divided areas is small. It gets smaller. If the size of the divided area is determined so that the change in the spectral transmission characteristic of the filter from one end to the other end in the divided area is sufficiently small, the filter spectral transmission characteristic in the divided area is constant. However, the effect on the image quality is reduced.

  For example, when the spectral transmission characteristics of the filter change gently with respect to the change in image height, it is possible to increase the divided area (reduce the number of divisions). When dividing an area, it is possible to increase the size of the divided area in an unequal division, that is, an area where the change in spectral transmission characteristics is small, and to reduce the size of the divided area in an area where the change in spectral transmission characteristics is large. In the following description, it is assumed that the area division processing unit 620 performs area division processing by the equal division method.

  The total spectral sensitivity characteristic deriving unit 626 outputs the spectral sensitivity distribution (spectral sensitivity characteristic corresponding to each pixel on the image area) on the image areas of the image sensors 218 and 220 output from the sensitivity nonuniformity deriving unit 614. The total spectral sensitivity characteristic corresponding to each pixel is obtained from the spectral transmission characteristics of the filters 214 and 216 corresponding to each pixel in the image area output from the filter characteristic change amount deriving unit 618, and representative characteristic information generation is performed. To the unit 628A. Here, the total spectral sensitivity characteristic means a value obtained by multiplying the spectral sensitivity characteristic of each pixel in the image area by the spectral transmission characteristic corresponding to the pixel. That is, as a result of receiving the light transmitted through the filters 214 and 216 by each pixel in the image area, the total spectral sensitivity characteristic equivalently indicates what spectral sensitivity characteristic has.

  FIG. 7A is a diagram conceptually illustrating an internal configuration example of the input profile information 700 generated by the input profile information generation unit 622. FIG. 7B conceptually shows data for specifying spectral sensitivity characteristics when the imaging apparatus 100 has 6 bands of spectral sensitivity. In FIG. 7C, the image area is divided into 108 divided regions by the region dividing processing unit 620, which is 12 in the X direction (direction along the long side of the image) and 9 in the Y direction (direction along the short side of the image). It is a figure which shows notably a mode that it is performed.

  In the example shown in FIG. 7A, the input profile information 700 includes information on the lens type, the set focal length, the number of bands, the number of spectral sensitivity patterns, the number of wavelength dimensions, the spectral sensitivity pattern, the number of X direction regions, and the number of Y direction regions. , A spectral sensitivity pattern designation table, a spectral sensitivity correction coefficient for each region, bias / tone curve data, subject characteristic information, and photographing illumination information.

  The lens type information includes information related to the focal length of the photographing lens. The lens type information may also include information (for example, a serial number) that can uniquely identify each lens. When the photographing lens is a so-called zoom lens, the lens type information preferably includes information on the focal length at the wide-angle end and the telephoto end, or information on the focal length and zoom ratio at the wide-angle end. In the example shown in FIG. 7A, the lens type information indicates that the photographing lens is a zoom lens having a focal length range of 12 mm to 60 mm, and the serial number is DDD012345. The lens type information can be input from the lens information acquisition unit 602 via the peripheral light attenuation deriving unit 610.

  The set focal length information indicates how many mm the input profile information 700 is set to the focal length of the photographing lens. In the example of FIG. 7A, the set focal length information is 15, which indicates that the input profile information 700 corresponds to that when the set focal length of the photographing lens is 15 mm.

  The information on the number of bands indicates the number of bands of an image that can be output from the imaging apparatus. In the example of FIG. 7A, the information on the number of bands is 6, which indicates that 6-band image data can be output.

Each information of the number of spectral sensitivity patterns, the number of wavelength dimensions, the spectral sensitivity pattern, the number of X direction regions, the number of Y direction regions, and the spectral sensitivity pattern designation table will be described. In the present invention, the representative characteristic information generation unit 628A is defined by the number of X direction divisions (this number is referred to as n _X ) and the number of Y direction divisions (this number is referred to as n _Y ) determined by the region division processing unit 620. Information for creating an input profile for each of the n _X × n _Y divided regions is generated. In FIG. 7A, the value (12) corresponding to the above n _X is stored as the number of X direction regions, and the value (9) corresponding to the above n _Y is stored as the number of Y direction regions. That is, the number of the divided regions, the 108 is the product of the n _X, n _Y. By the way, a plurality of pixels exist in each divided region, and the total spectral sensitivity characteristic is different for each pixel. Based on the total spectral sensitivity corresponding to each of a plurality of pixels existing in the divided area, the representative characteristic information generation unit 628A generates information on the total spectral sensitivity characteristic representing the divided area. In this specification, this information is referred to as “divided region representative characteristic information”.

  Spectral sensitivity characteristics representing each of the 108 divided regions can be defined by data as conceptually shown in FIG. 7B. This spectral sensitivity characteristic is desirably the total spectral sensitivity characteristic described above. In the present embodiment, the photographing apparatus 100 has a six-band spectral sensitivity by combining a B, G, and R on-chip color filter and two types of filters 214 and 216 having different spectral transmission characteristics. The spectral sensitivity of 6 bands can be represented by 6 spectral sensitivity profiles as illustrated in FIG. 7B. In FIG. 7B, the spectral sensitivity profile is defined as a sensitivity corresponding to 41 wavelength points in a wavelength band of 380 nm to 780 nm (that is, black circles are data defining spectral sensitivity in the graph of FIG. 7B). (Representing a point corresponding to).

  The number of wavelength dimensions in FIG. 7A indicates how many spectral points define the spectral sensitivity in the visible light band (band of wavelengths from 380 nm to 780 nm). For example, when the spectral sensitivity is defined at wavelength points of 1 nm intervals, the number of wavelength dimensions is (780-380) / 1 + 1 = 401, and when the spectral sensitivity is defined at wavelength points of 5 nm intervals, the number of wavelength dimensions Becomes (780-380) / 5 + 1 = 81. In the example of FIG. 7B, since the spectral sensitivity is defined at intervals of 10 nm, the number of wavelength dimensions is 41.

  In the above description, the case where the wavelength band is set by default in the range of 380 nm to 780 nm has been described. However, the spectral sensitivity profile may be defined over a wider wavelength range or may be defined over a narrow wavelength range. In that case, for example, by including information on the shortest wavelength, wavelength interval, and number of wavelength dimensions in the input profile, a spectral sensitivity profile in an arbitrary wavelength band (shortest wavelength-shortest wavelength + wavelength interval x wavelength dimension number) is defined. It becomes possible to do.

  Information on the number of spectral sensitivity patterns in FIG. 7A will be described. As described above, in this embodiment, spectral sensitivity information conceptually shown in FIG. 7B is assigned to each of the 108 divided regions. For example, 108 types of tiles to be applied to each of the 108 divided areas are required. However, by allowing a certain amount of error, as the spectral sensitivity characteristics corresponding to the 108 divided areas, for example, the divided areas (1, 1), (2, 1), (12, 1) in FIG. The spectral sensitivity characteristics in (1, 3), (11, 5), (12, 9) can be handled substantially the same. That is, the same type of tile can be applied to several divided areas among the 108 divided areas. By doing so, the types of tiles can be reduced. Accordingly, the capacity of the input profile can be greatly reduced.

  Hereinafter, how to handle the total spectral sensitivity characteristic in which the spectral sensitivity characteristics of the filters 214 and 216 are added to the spectral sensitivity characteristics of the image sensors 218 and 220 will be described. Prepared distribution of the total spectral sensitivity characteristic of the whole image area (the total spectral sensitivity characteristic corresponding to each pixel in the image area), and a permission information capacity of the obtained image quality and input profile, whatever the spectral sensitivity pattern in advance It is possible to incorporate a process for determining whether or not to be performed into an algorithm of a program executed by the camera CPU 102.

  At this time, when a higher image quality is required and the amount of information of the input profile information 700 is allowed to be large, the number of spectral sensitivity patterns may be equal to the number of divided regions (108 in this embodiment). it can. On the other hand, when the image quality is expected to be reduced, but it is required to reduce the information amount of the input profile information 700, the number of spectral sensitivity patterns may be reduced. Of course, if the change is slow when looking at the distribution of the overall spectral sensitivity characteristics over the entire image area, the number of spectral sensitivity patterns can be reduced without causing a reduction in image quality. In an extreme example, the number of spectral sensitivity patterns may be one, which is the minimum number.

  The information on the number of spectral sensitivity patterns in FIG. 7A indicates how many sets of information defining the pattern of the total spectral sensitivity characteristic are included in the input profile information. In the example shown in FIG. 7A, the number of spectral sensitivity patterns is 30. That is, as shown in FIG. 7A, 6-band sensitivity 1, 6-band sensitivity 2,..., 6-band sensitivity 30 and 30 types of spectral sensitivity patterns (30 types of tiles) are included in the input profile information. means.

  Among the 30 types of spectral sensitivity patterns, one of the spectral sensitivity patterns is assigned to each of the divided regions (1, 1) to the divided regions (12, 9). That is, the spectral sensitivity pattern designation table in FIG. 7A is information for designating spectral sensitivity patterns (any of 6-band sensitivity 1, 6-band sensitivity 2,..., 6-band sensitivity 30) to be assigned to each divided region. Is included.

  In the present specification, information including the number of spectral sensitivity patterns, the number of wavelength dimensions, the spectral sensitivity pattern, the number of X direction regions, and the number of Y direction regions shown in FIG. 7A is referred to as divided region representative characteristic information. In addition, information including a spectral sensitivity pattern designation table, that is, a table that applies which divided area representative characteristic information to each divided area is referred to as representative characteristic designation information. Further, in this specification, a process of assigning the same spectral sensitivity pattern to a plurality of divided regions by reducing the number of spectral sensitivity patterns obtained by the same number as the number of divided regions is referred to as “spectral sensitivity characteristics”. This is referred to as “information reduction processing”. Details of the spectral sensitivity characteristic information reduction processing will be described later.

  As the spectral sensitivity correction coefficient for each area, a set of coefficient sets is formed by six coefficients arranged side by side, and this set of coefficient sets is applied to one divided area. In the example shown in FIG. 7A, as described above, any of the 30 sets of 6-band sensitivities is applied to each of the 108 divided regions, but in the applied spectral sensitivity pattern, Each of the six spectral sensitivity characteristics (6-band sensitivity) included is multiplied by the above six coefficients. The spectral sensitivity correction coefficient for each region is generated based on information output from the peripheral light attenuation deriving unit 610. Accordingly, one coefficient may be uniformly multiplied for the six-band sensitivity. However, by having independent coefficients for each of the 6-band sensitivities as described above, it is possible to define the spectral sensitivity characteristics in each divided region more finely as described below.

  Now, six spectral sensitivity characteristics (corresponding to the spectral sensitivity profiles 1, 2,..., 6 in FIG. 7B) defined by any one of the six-band sensitivity 1 to the six-band sensitivity 30 are displayed from the short wavelength side. 1, spectral sensitivity characteristics 2,..., Spectral sensitivity characteristics 6. Of the spectral sensitivity correction coefficients for each region illustrated in FIG. 7A, the top six coefficients (1.00026, 1.0049, 0.9971, 0.9961, 1.0028, 1.0084) are divided. Multiplied by spectral sensitivity characteristic 1, spectral sensitivity characteristic 2,..., Spectral sensitivity characteristic 6 in the spectral sensitivity pattern (6 band sensitivity 1 is applied in the example of FIG. 7A) applied to region (1, 1).

  That is, the component ratio of the spectral sensitivity characteristic 1, the spectral sensitivity characteristic 2,..., The spectral sensitivity characteristic 6 can be freely changed by these six coefficients. For example, increasing or decreasing these six coefficients in the same way is equivalent to increasing or decreasing the sensitivity over the entire visible light region. When only a specific coefficient is increased or decreased, only the spectral sensitivity characteristic multiplied by the coefficient can be increased or decreased.

  Similarly, the six coefficients (0.9973, 1.050501.0030, 0.9961, 1.0029, 0.9915) in the second stage are spectral sensitivity patterns (2 and 1) applied to the divided region (2, 1). In the example of FIG. 7A, 6-band sensitivity 2 is applied), and spectral sensitivity characteristic 1, spectral sensitivity characteristic 2,..., Spectral sensitivity characteristic 6 are multiplied.

  As described above, as the spectral sensitivity correction coefficient for each area, the divided area (1, 1), the divided area (2, 1),..., The divided area (12, 1), the divided area (1, 2), the divided area. Six coefficients are included in the input profile information 700 for each of (2, 2),..., And the divided regions (12, 9). That is, the spectral sensitivity correction coefficient for each region includes 6 (number of bands) × 12 (number of regions in the X direction) × 9 (number of regions in the Y direction) = 648 pieces in the example of FIG. 7A.

  Summarizing the contents described above, among the above-mentioned factors of unevenness, the spectral sensitivity pattern (6-band sensitivity 1) applied to each divided region with respect to the sensitivity variation of the image sensor and the incident angle dependence of the spectral transmission characteristics of the filter. 6 band sensitivity 30) can be changed. On the other hand, among the above-mentioned unevenness factors, the peripheral dimming characteristics of the photographing lens and the decrease in the amount of light obliquely incident on the image sensor are areas corresponding to changes in the amount of light incident on the image sensor. This can be dealt with by changing the spectral sensitivity correction coefficient. Then, the number of spectral sensitivity patterns, the spectral sensitivity pattern, the number of X direction regions, and the number of Y direction regions are determined according to the degree of unevenness and the required image quality.

  When the spectral transmission characteristic (CCI) of the photographing lens is deviated from the standard value, or when a color correction filter or the like is attached to the photographing lens and the CCI of the filter deviates from the standard value, This can be dealt with by changing the spectral sensitivity correction coefficient for each region. That is, when the CCI of the photographing lens is deviated from the standard value, the color of the entire image is deviated. In such a case, among the spectral sensitivity correction coefficients for each region described with reference to FIG. 7A, for example, the coefficient multiplied by the spectral sensitivity characteristic 6 described above (108 data 1 in the rightmost column in FIG. 7A 1 .0084, 0.9915, 0.9914,..., 0.9917) can be uniformly increased or decreased to correct the color of the entire image.

  The input profile information 700 described above with reference to FIG. 7A corresponds to the case where the photographing lens 110 is set to a specific focal length (the set focal length is 15 mm in the example of FIG. 7A). It is. By the way, when the photographing lens 110 is a zoom lens or when the amount of fluctuation of the lens position due to focus adjustment is large like a macro lens, the following is photographed in the input profile information illustrated in FIG. 7A. It may change according to the state of the photographing lens 110 at the time. That is, the spectral sensitivity pattern, the number of X direction regions, the number of Y direction regions, the spectral sensitivity pattern designation table, and the spectral sensitivity correction coefficient for each region vary depending on the situation of the photographing lens 110 at the time of photographing. That is, in the input profile information 700, the divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region may vary depending on the photographing situation depending on the type of the photographing lens 110.

  In the input profile information 700 described above, the divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region are obtained when the photographing lens 110 is attached to the photographing apparatus 100 and the photographing apparatus 100 is turned on. It is desirable that the input profile information generation unit 622 generates the data. For example, when the photographic lens to be mounted is a zoom lens capable of changing the focal length in the range of 12 mm to 60 mm, the input profile information generation unit 622 has a plurality of sets of divided region representative characteristics as illustrated in FIG. 7D. Information, representative characteristic designation information, and spectral sensitivity correction coefficients 702-1, 702-2, ..., 702-n for each region are generated and stored in the memory 205. The plurality of sets of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficients 702-1, 702-2,... 702-n for each region have the smallest aperture value for the taking lens 110 (open). (Aperture).

  When the setting aperture of the photographic lens 110 changes, the exit pupil diameter changes. Peripheral dimming is relatively greatly affected by changes in the exit pupil diameter. Therefore, by changing the spectral sensitivity correction coefficient for each region (6 × 108 data in the example of FIG. 7A) in the input profile information 700 shown in FIG. 7A, the influence of the setting aperture of the taking lens 110 is reduced. It becomes possible.

  The information aggregation processing unit 628C obtains information related to the set focal length and the set aperture of the shooting lens from the shooting information acquisition unit 604. The information aggregation processing unit 628C then divides the area representative characteristic information corresponding to the focal length set at the time of shooting, representative characteristic designation information, and spectral sensitivity correction coefficients for each area (702-1, 702-2, ..., 702). -N) is selected, and the spectral sensitivity correction coefficient for each region included in the information is changed according to the set aperture value of the photographic lens 110.

  The information aggregation processing unit 628C generates the input profile information 700 by adding the bias / tone curve data, the subject characteristic information, and the photographing illumination information to the information generated as described above, and outputs the input profile information 700 to the data format processing unit 624. . The bias / tone curve data, subject characteristic information, and photographing illumination information are as described above with reference to FIG. 6A. In the present embodiment, the multiband image data is described as being 6-band image data. Correspondingly, the bias correction information can include a bias value applied to each image data of the first to sixth bands. Similarly, the tone curve correction information can include a look-up table (LUT) for correcting the tone characteristics of the image data of the first to sixth bands.

  With reference to FIG. 6A, the data format processing unit 624 adds the input profile information output from the input profile information generation unit 622 to the multiband image data output from the image processing unit 206. The data format processing unit 624 performs processing for recording the multiband image data as original image data in the storage medium 240 in a predetermined file format.

  FIG. 8A shows generation of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each region, which is executed by the camera CPU 102 when the photographing lens 110 is attached to the photographing apparatus 100 and the power is turned on. It is a flowchart explaining a procedure.

  In S800, the CPU 102 extracts lens type information (eg, focal length range of lens, serial number, etc.), exit pupil position information, and peripheral dimming information from the acquired lens information. In step S <b> 802, the CPU 102 reads from the memory 205 information regarding the number of bands (= b) of the image sensors 218 and 220, the number of wavelength dimensions, the filter spectral transmission characteristics, and the spectral sensitivity unevenness of the image sensors 218 and 220. In step S <b> 804, the CPU 102 reads out information on the number of filters (= f) from the memory 205 and determines the number of bands (= b × f).

  In step S806, the CPU calculates a filter incident angle characteristic from the image height obtained from the pixel position in the image area and the exit pupil position information extracted in step S800, and determines the number of X-direction regions and the number of Y-direction regions. .

  In step S808, the CPU 102 obtains the spectral transmission characteristics of the filters 214 and 216 corresponding to the respective pixels in the image area, and calculates the spectral transmission characteristics on the image areas of the image sensors 218 and 220 (each of the pixels on the image area). To obtain the total spectral sensitivity characteristic. This total spectral sensitivity characteristic is summarized as corresponding to each pixel in one image obtained by processing image signals output from the two image sensors 218 and 220.

  The above will be described. One total spectral sensitivity characteristic (referred to as total spectral sensitivity characteristic A) is obtained from the combination of the spectral transmission characteristic of the filter 214 and the spectral sensitivity characteristic distribution of the image sensor 218. The spectral transmission characteristic of the filter 216 and the image sensor Another total spectral sensitivity characteristic (referred to as total spectral sensitivity characteristic B) is obtained from the combination with the distribution of 220 spectral sensitivity characteristics. However, one 6-band image data is obtained by synthesizing image signals obtained from the two image sensors 218 and 220. Therefore, the CPU 102 synthesizes the total spectral sensitivity characteristic A and the total spectral sensitivity characteristic B to generate one total spectral sensitivity characteristic.

  In the above description regarding the processing of S808, the total spectral sensitivity characteristic A is obtained from a combination of the spectral transmission characteristic of the filter 214 corresponding to each pixel in the image area and the spectral sensitivity characteristic distribution of the image sensor 218. explained. Further, it has been described that the total spectral sensitivity characteristic B is obtained from a combination of the spectral transmission characteristic of the filter 216 corresponding to each pixel in the image area and the spectral sensitivity characteristic distribution of the image sensor 220. In these respects, the overall sensitivity characteristic A reflects only the variation in spectral transmission characteristics of the filter 214 corresponding to each pixel in the image area, and the overall spectral sensitivity characteristic B represents the filter 216 corresponding to each pixel in the image area. Only the spectral transmission characteristic may be reflected.

  That is, the total spectral sensitivity characteristic A is the spectral transmission characteristic of the filter 214 corresponding to each pixel in the image area and the representative spectral sensitivity characteristic of the image sensor 218 (for example, the spectral sensitivity characteristic near the center of the image area). It may be obtained from a combination. The total spectral sensitivity characteristic B may be obtained from a combination of the spectral transmission characteristic of the filter 216 corresponding to each pixel in the image area and the representative spectral sensitivity characteristic of the image sensor 218. The reason is that the influence due to the unevenness of the spectral sensitivity characteristic due to the variation in the manufacturing process of the image sensor is generally smaller than the influence due to the change in the incident angle when the subject light passes through the filter. Further, the unevenness of the spectral sensitivity characteristics of the image sensor can be reduced to a negligible level by improving the manufacturing process.

  In S810, the CPU 102 determines the representative characteristic of the total spectral sensitivity characteristic for each of a plurality of (108 in the example of FIG. 7C) divided areas determined from the number of X-direction areas and the number of Y-direction areas determined in S806. That is, the CPU 102 determines the representative characteristic corresponding to the divided area from the total spectral sensitivity characteristic corresponding to each pixel in the divided area (the total spectral sensitivity characteristic can be expressed by a 6-band spectral sensitivity profile). As a method for determining the representative characteristics, any method such as simple average, mode extraction, least square approximation, or the total spectral sensitivity characteristics of the pixel located at the center of the divided area can be used. is there.

  In S812, the CPU 102 analyzes representative characteristics of the total spectral sensitivity corresponding to each divided area (108 sets of total spectral sensitivity information in the example of FIG. 7), and performs the spectral sensitivity characteristic information reduction process described above. That is, the CPU 102 determines the number (n) of spectral sensitivity patterns and defines the spectral sensitivity profile corresponding to the spectral sensitivity patterns 1, 2,..., N (6-band sensitivity 1, 6-band in FIG. 7A). Sensitivity 2,..., Corresponding to 6 band sensitivity 30) is generated. In the example shown in FIG. 7A, n = 30. The number n of spectral sensitivity patterns can be determined based on variations in total spectral sensitivity within one screen, image quality desired by the photographer, data capacity available for input profile information, and the like.

  An example of the spectral sensitivity characteristic information reduction process performed by the CPU 102 in S812 will be described with reference to FIG. 7C. In the processing example described here, color difference is used. When the processing of S810 is completed, the representative characteristic of the total spectral sensitivity characteristic is determined corresponding to each of the 108 divided regions (1, 1), (2, 1), ..., (12, 9). Yes. Therefore, other divided areas (1, 2), (2, 1),..., (12) based on the total spectral sensitivity characteristics of the divided areas located near the center of the image area, for example, the divided areas (6, 5). , 9) can be compared by color difference.

  Specifically, the comparison is performed as follows. That is, the CPU 102 assumes that white light is transmitted through the filters 214 and 216 and is incident on the image areas of the image sensors 218 and 220, and obtains a signal value (which is obtained from the total spectral sensitivity characteristic of each divided region). (Referred to as virtual output signal values) [a1, a2,..., A6] are obtained for each divided region.

  Next, the CPU 102 determines the total spectral sensitivity of the divided areas located near the center of the image area (here, the divided areas (6, 5) are selected as the divided areas located near the center of the image area). A spectral reflectance estimation matrix [M] (described later) is calculated using the characteristics ([] means a matrix). At this time, [M] is convenient and does not need to be an accurate reflectance estimation matrix. For example, the imaging illumination spectrum may be calculated assuming illumination having flat energy in the entire wavelength band. Absent. Thereafter, the virtual output signal values [an1, an2,..., An6] of the divided areas located in the vicinity of the center are used as the reference, and the virtual output signal values [ a1, a2,..., a6], and a color difference corresponding to each divided area is obtained.

  There is known a method for calculating a color difference from a colorimetric value obtained by multiplying the spectral reflectance by illumination and applying a color matching function of human vision. The calculation method is also accurate as described above. It is not necessary, and it is only necessary to determine whether the colors are relatively different between the divided areas. In this embodiment, the distance on the xy chromaticity coordinates is obtained as a color difference. By this processing, 108 pieces of color difference information corresponding to 108 divided regions are obtained. As a matter of course, the color difference corresponding to the divided area (6, 5) is zero. Since the color difference does not include information regarding the direction on the chromaticity coordinate, the same color difference is not necessarily the same color. The method shown in this example assumes that the spectral transmission characteristic of the filter has a dominant influence on the total spectral sensitivity characteristic, and the color change spreads concentrically around the center of the image area, that is, the optical axis. This is an example.

  Subsequently, the CPU 102 obtains a difference between the maximum value and the minimum value of the 108 color differences, and determines the number n of spectral sensitivity patterns based on this difference (color difference distribution width). The CPU 102 classifies the 108 color differences into n classes, and determines the color difference representing each class from the color difference values existing in each class. Note that the class width may be equal width or unequal width as necessary, but in this embodiment, the class width is assumed to be equal. The color difference representing each class can be the color difference closest to the intermediate value of each class. With the above processing by the CPU 102, n values for the color difference can be selected. The CPU 102 determines a spectral sensitivity pattern of 6 band sensitivity 1, 6 band sensitivity 2,... Illustrated in FIG. 7A based on the total spectral sensitivity characteristics corresponding to these n color differences.

  In S814, the CPU 102 generates a table for designating which spectral sensitivity pattern is applied to each divided region among the 6-band sensitivity 1, the 6-band sensitivity 2,..., The 6-band sensitivity n generated in S810. This table is the spectral sensitivity pattern designation table shown in FIG. 7A.

  A specific example of the process in S814 will be described. By the processing in S812, the CPU 102 can grasp the color difference in each divided area and the color difference corresponding to the selected n spectral sensitivity patterns. The CPU 102 selects a spectral sensitivity pattern having a color difference closest to the color difference of each divided area from the n spectral sensitivity patterns. By performing this selection process on all the divided areas, the CPU 102 generates a spectral sensitivity pattern designation table.

  In S816, the CPU 102 performs processing for obtaining a spectral sensitivity correction coefficient for each region based on the peripheral light attenuation characteristics. The spectral sensitivity correction coefficient for each region shown in FIG. 7A is obtained by the processing of S816. When the CPU 102 obtains the spectral sensitivity correction coefficient for each region, the coefficient can be obtained as follows. In the example shown in FIG. 7A, the number of spectral sensitivity patterns that originally were 108 sets is reduced to 30 sets.

  The spectral sensitivity pattern assigned to a given divided region by the spectral sensitivity characteristic information reduction process described above may be slightly different from the spectral sensitivity pattern originally possessed by the divided region. This means that an error may occur in the color reproduction characteristics in the divided area. On the other hand, as described above, the spectral sensitivity correction coefficient has six values that can be multiplied with each of the six spectral sensitivity profiles as shown in FIG. 7B, for example. Therefore, the spectral sensitivity correction coefficient obtained in S816 not only reduces the influence of the peripheral light attenuation characteristics, but also the component ratio of the six spectral sensitivity profiles as if the sound quality is changed by operating the graphic equalizer with the audio amplifier. Can be adjusted. As a result, it is possible to reduce errors that may be caused by reducing the number of spectral sensitivity patterns. The process in S816 corresponds to the process performed by the correction coefficient generation unit 628B in FIG. 6B.

  In step S <b> 818, the CPU 102 determines whether generation of a set of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each region corresponding to all focal lengths is completed. While this determination is denied, the CPU 102 repeats the processing from S800 to S818. If the determination in S818 is affirmed, the CPU 102 completes the process of generating divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each region. 8A described above, a plurality of sets of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficients for each region 702-1, 702-2,... 702 as illustrated in FIG. -N is recorded in the memory 205.

  FIG. 8B shows the input profile information executed by the camera CPU 102 when shooting is performed by the shooting apparatus 100 following the above-described procedure for generating the divided area representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each area. It is a flowchart explaining a production | generation process procedure. The processing shown in FIG. 8B is executed after the image capturing is performed by the image capturing apparatus 100 and the digital image signals are stored in the first buffer 202 and the second buffer 204.

  In step S830, the CPU 102 acquires information related to the set focal length and the set aperture set by the taking lens 110 at the time of shooting. In step S832, the CPU 102 issues a control signal to the image processing unit 206 so as to start image processing.

  In step S834, the CPU 102 includes a plurality of sets of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficients 702-1, 702-2,. To select the one corresponding to the set focal length. In step S836, the CPU 102 performs processing for deriving the peripheral dimming characteristics based on the information on the set aperture.

  In S838, the CPU 102 obtains the spectral attenuation correction coefficient for each area included in the divided area representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each area selected in S834, and the peripheral attenuation characteristic derived in S836. Perform the correction process.

  In step S840, the CPU 102 includes information on the lens type, the set focal length, and the number of bands, bias / tone curve data, subject characteristic information, divided area representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each area. Input profile information is generated by adding shooting illumination information and the like.

  In S842, the CPU 102 adds the input profile information generated as described above to the image data, and records it in the storage medium 240 in a format according to the “natural vision video data file format”. The natural vision video data file format is described in detail in Non-Patent Document 1.

  Each time shooting is performed by the imaging apparatus 100, the CPU 102 performs the processing described with reference to the flowchart of FIG. 8B. As a result, input profile information is generated, and this input profile information is added to the multiband image data and stored in the storage medium 240 as original image data. When the photographing lens 110 is replaced with another type during photographing, the processing described with reference to the flowchart of FIG. 8A is performed by the CPU 102, and new divided region representative characteristic information, representative characteristic designation information, In addition, a spectral sensitivity correction coefficient for each region is generated.

  In the following, the storage medium 240 is taken out from the photographing apparatus 100 and attached to the information processing apparatus 140, and the original image data in the storage medium 240 is read by the information processing apparatus 140 to perform input profile creation processing and color conversion processing. Shall be.

  FIG. 9 is a block diagram illustrating a schematic configuration of the color conversion processing unit 900. In the present embodiment, the color conversion processing unit 900 is described as configured by interpreting and executing software for color conversion processing by the information processing apparatus 140, but is configured by a dedicated logic circuit, ASIC, or the like. Also good. In this embodiment, the information processing apparatus 140 is described as a computer. However, the present invention is an image reproducing apparatus, a video recorder, a set top box (STB), a printer, and a portable data storage with an image display function. It can be applied to devices, mobile phones and the like.

  The color conversion processing unit 900 separates multiband image data and input profile information included in the input original image data D, and creates an input profile based on the input profile information. The color conversion processing unit 900 performs color conversion processing on the multiband image data using the created input profile, and outputs the processed image data to an image output device 950 such as a monitor display device or a printer.

  The color conversion processing unit 900 includes an input profile operation unit 910, an output profile operation unit 920, and an input profile creation unit 930. The input profile application unit 910 includes a tone correction unit 912, a region division unit 914, and a matrix calculation unit 916 for each region. In FIG. 9, the matrix calculation unit for each region is described as “MTX calculation unit for each region”.

  The input profile creation unit 930 includes a bias / tone curve correction information acquisition unit 932, a region division information acquisition unit 934, and a spectral reflectance estimation matrix creation unit for each region (in FIG. 9, “spectral reflectance estimation MTX for each region”). 936, a system function calculation unit 938 for each region, a spectral sensitivity creation unit 940 for each region, and a photographing illumination information acquisition unit 942.

  The bias / tone curve correction information acquisition unit 932 acquires the bias / tone curve correction information from the input profile information and outputs it to the tone correction unit 912. In this embodiment, the case where the multiband image data is 6-band image data is described, and the bias correction information includes a bias value applied to each image data of the first band to the sixth band. Similarly, the tone curve correction information includes a look-up table (LUT) that corrects the tone characteristics of the image data of the first to sixth bands. The area division information acquisition unit 934 acquires information on the number of areas (the number of X-direction areas and the number of Y-direction areas described with reference to FIGS. 7A and 7B) from the input profile information, and outputs the acquired information to the area division unit 914.

  The spectral sensitivity creation unit 940 for each region acquires the spectral sensitivity pattern, the spectral sensitivity designation table, and the spectral sensitivity correction coefficient information for each region described with reference to FIG. 7A from the input profile information. Create sensitivity. Specifically, each spectral sensitivity pattern (6-band sensitivity 1, 6-band sensitivity 2,...) Designated corresponding to each region is multiplied by the spectral sensitivity correction coefficient for each region, and the spectral sensitivity characteristic for each region is obtained. Processing to calculate is performed.

  The imaging illumination information acquisition unit 942 acquires imaging illumination information from the input profile information. The system function calculation unit 938 for each region calculates the system function for each region by multiplying the spectral sensitivity for each region created by the spectral sensitivity creation unit 940 for each region by imaging illumination information. When the subject is illuminated by a single light source at the time of shooting, there is one type of shooting illumination information that is multiplied by the spectral sensitivity for each region. When the subject is illuminated by a plurality of light sources at the time of shooting, the input profile information includes illumination information corresponding to each of the divided areas. In this case, the system function calculation unit for each region calculates the system function for each region by multiplying the spectral sensitivity for each region created by the spectral sensitivity creation unit 940 for each region by the imaging illumination information for each region.

The spectral reflectance estimation matrix creation unit 936 for each region uses the system function for each region calculated by the system function calculation unit 938 for each region, and the subject characteristic information included in the input profile information, and the spectral reflection for each region. A rate estimation matrix M is created. The spectral reflectance estimation matrix M can be created using the following equation. In addition, the brackets [] attached to the reference sign in the following formula means a matrix. [] ^T means a transposed matrix, and [] ^-1 means an inverse matrix.

[M] = [R] _rr [H] ^T ([H] [R] _rr [H] ^T [R] _nn ) ⁻¹ ) Formula (1)

In the above formula (1), [R] _nn denotes a correlation matrix of the noise can be represented for example by the following equation.

[R] _rr is called object characteristic information and is a correlation matrix of [r] and is expressed by the following equation. Note [r] _i is that the spectral reflectance characteristics of a plurality of samples.

[H] means a system function and is represented by the following equation.

_{[H] = h k (λ} ) I (λ)
h _k (λ): spectral sensitivity for each region (k = 1 to N)
I (λ): Shooting illumination information (N × N diagonal matrix)
The input profile operation unit 910 includes a tone correction unit 912, a region division unit 914, and a matrix calculation unit for each region (indicated as “MTX calculation unit for each region” in FIG. 9) 916. The tone correction unit 912 corrects the multiband image data using the bias and tone curve correction information output from the bias / tone curve correction information acquisition unit 932. The area dividing unit 914 divides the multiband image data based on the number of X direction areas and the number of Y direction areas output from the area division information acquisition unit 934.

  Note that “dividing the multiband image data” described here means that the image data itself is not actually divided, but is divided into sections according to the divided areas defined by the photographing apparatus 100. The matrix calculation unit 916 for each region applies the spectral reflectance estimation matrix corresponding to the region to the image data in the region divided by the region dividing unit 914, and the spectral reflectance image for each divided region. Generate data.

  The output profile of the image output device 950 is applied to the spectral reflectance image data generated by the input profile operation unit 910 by the output profile operation unit 920 and output to the image output device 950. In the embodiment of the present invention, the image output device 950 can be a printer or a monitor display device.

  10A and 10B are flowcharts for explaining the procedure of the color conversion process executed by the information processing apparatus 140. As described above, the color conversion process can be executed by software, a dedicated logic circuit, or the like, but the software stored in the information processing apparatus 140 is executed by the information processing apparatus 140. May be. This software may be, for example, image processing software capable of browsing and processing images, or may be video playback software when the multiband image data (original image data) to be processed is a video. In the following, image processing software is executed by the information processing device 140, multiband still image data (original image data) recorded in the storage medium 240 is processed, and output to an image output device 950 such as a monitor display device. It will be described as being done.

  In step S <b> 1002, the information processing apparatus 140 performs processing for reading original image data from the storage medium 240 into the storage device in the information processing apparatus 140.

  In step S1004, the information processing apparatus 140 performs processing for acquiring the input profile information 700 from the original image data read in step S1002. The information processing apparatus 140 performs processing for acquiring bias / tone curve correction information in S1006 from the input profile information 700 acquired in S1004 and acquiring region division information in S1008.

  In step S <b> 1010, the information processing apparatus 140 performs processing for acquiring the spectral sensitivity pattern, the spectral sensitivity pattern designation table, and the spectral sensitivity correction coefficient for each region from the input profile information 700.

  The information processing apparatus 140 performs processing for obtaining shooting illumination information in S1012 and subject characteristic information in S1014 from the input profile information 700, respectively. In S1016, the information processing apparatus 140 derives a system function Hn for each region by multiplying the spectral sensitivity pattern specified in the spectral sensitivity pattern specification table by the spectral sensitivity correction coefficient for each region.

In step S <b> 1018, the information processing apparatus 140 estimates the spectral reflectance for each area using the subject characteristic information [R] _rr , the system function [H] _n for each area, and the imaging illumination spectrum (imaging illumination information) I (λ). The matrix [M] n is derived.

Through the processing up to S1018 described above, the spectral reflectance estimation matrix [M] n (n: 1, 2,..., N _X × n _Y ) corresponding to each divided region is obtained. Subsequently, color conversion processing is performed on the pixel signal values corresponding to each pixel in the image data by the processing from S1020 to S1026 described below with reference to FIG. 10B.

  In step S1020, the information processing apparatus 140 performs bias / tone correction processing. That is, a process of subtracting the bias value from each of the pixel signal values of all the pixels is performed, and further, a process of converting to a linear signal value by the tone characteristic information is performed. In the present embodiment, six linear signal values are obtained corresponding to one pixel.

  In step S1022, the information processing apparatus 140 identifies a divided region to which a pixel to be processed (hereinafter referred to as a “processing target pixel”) belongs. In S1024, the information processing apparatus 140 generates a spectral reflectance image signal by multiplying the spectral reflectance estimation matrix corresponding to the divided region to which the pixel to be processed belongs by the linear signal value obtained in S1020. This spectral reflectance image signal is an image signal that is device independent (device independent).

  In S1026, the information processing apparatus 140 determines whether or not the generation of the spectral reflectance image signal corresponding to all the pixels in the processing target image data (original image data) has been completed, and while this determination is negative, S1020. To S1026 are repeated. If the determination in S1026 is affirmative, the process proceeds to S1028. In step S <b> 1028, the information processing apparatus 140 causes the output profile of the image output apparatus 950 to act on the spectral reflectance image signal to generate a device-dependent (device dependent) image signal, and outputs it to the image output apparatus 950. To do. In the process of S1028, it is also possible to consider the spectral characteristics of illumination light that illuminates the environment in which the image output device 950 exists (this is referred to as “observation illumination light” or “environment light”).

  As described above, according to the photographing apparatus 100 and the information processing apparatus 140 of the first embodiment of the present invention, a single input profile is not uniformly applied to the entire image. Instead, divided areas are determined corresponding to the above-described unevenness, and input profiles corresponding to the respective divided areas are applied. Then, input profile information is added to the original image data generated by the photographing apparatus 100 instead of the input profile corresponding to each pixel.

  Therefore, it is possible to suppress the increase in the capacity of the original image data while enabling more faithful color reproduction. In addition, since it is not necessary to perform processing for creating an input profile in the photographing apparatus 100, the processing load on the photographing apparatus 100 can be reduced. By suppressing the increase in the capacity of the original image data, the number of frames that can be recorded in the storage medium 240 is increased, and the time required for transferring the original image data to another apparatus via some communication means is shortened. It becomes possible to do. In addition, by reducing the processing load on the image capturing apparatus 100, the continuous shooting speed of the image capturing apparatus is increased, and the time from when one frame is captured until the next image capturing operation becomes possible is shortened. It becomes possible to improve.

  In the above, an example in which the photographing apparatus 100 is of the photographing lens exchange type has been described, but the photographing lens may be fixed to the photographing apparatus main body. The photographing apparatus 100 may be capable of photographing not only a still image but also a moving image. In the case of a moving image, it is possible to add one input profile information to a series of moving image files. In addition, when there is a change in shooting conditions, such as zooming operation of the shooting lens, shooting distance, etc. during movie shooting, it is also possible to embed input profile information at any time in the movie file where the shooting conditions have changed. . Alternatively, only information capable of specifying shooting conditions may be embedded in a moving image file, and input profile information corresponding to a plurality of shooting conditions may be added to the beginning or end of a series of moving image files.

  In the photographing apparatus 100 described with reference to FIG. 2, the subject image formed by the photographing lens 110 is divided into two by the beam splitter 230, and each of the divided subject light passes through the filters 214 and 216 and the imaging element 218. , 220. However, the present invention is not limited to this.

  For example, the photographing apparatus 100 may have a so-called single-plate configuration having one image sensor including an on-chip color filter, and may obtain multiband image data for one screen by two photographings. It is possible to switch the filter when shooting twice. In that case, the front side of the photographing lens 110 (between the subject and the light incident surface of the photographing lens 110), the inside of the photographing lens 110, and the rear side of the photographing lens 110 (between the light exit surface of the photographing lens 110 and the image sensor). ), The filter can be switched at any position.

  In addition, even in an imaging device having a so-called three-plate configuration in which the imaging device does not have an on-chip color filter and has a dichroic prism that splits subject light into light in three spectral bands and three monochrome imaging devices. Multi-band image data for one screen can be obtained by two shootings. In that case, the filter is provided at any position on the front side of the photographic lens 110, in the photographic lens 110, and on the rear side of the photographic lens 110 (between the light exit surface of the photographic lens 110 and the incident surface of the dichroic prism). It is possible to switch.

  In addition, in the method of switching filters when shooting twice using a single-plate or three-plate imaging device as described above, instead of mechanically switching the two filters, the spectral transmission characteristics are changed. It is also possible to use an electrically switchable filter. In that case, the spectral transmission characteristics can be electrically switched in conjunction with the two shootings.

− Second Embodiment −
In the first embodiment, the example in which the input profile information added to the multiband image data is generated inside the imaging apparatus 100 that performs imaging to generate multiband image data has been described. In contrast, in the second embodiment, the input profile information is generated by an information processing apparatus other than the imaging apparatus used for imaging.

  In the first embodiment, the example in which the input profile information described with reference to FIG. 7A is added to the multiband image data generated by the imaging apparatus 100 has been described. That is, the input profile information 700 includes the number of bands, the number of spectral sensitivity patterns, the number of wavelength dimensions, the spectral sensitivity pattern, the number of X direction regions, the number of Y direction regions, the spectral sensitivity pattern designation table, the spectral sensitivity correction coefficient for each region, and the bias. An example in which tone curve data, subject characteristic information, and photographing illumination information are included has been described.

  On the other hand, in the second embodiment, as shown in FIG. 11, the input profile information added to the multiband image data generated by the photographing apparatus 100A is an image such as a camera ID or a lens ID. It includes input device ID information, shooting setting information, subject characteristic information, and shooting illumination information. When photographing is performed using the photographing apparatus 100A, as an example, the original image data to which the input profile information 700A as illustrated in FIG. 13 is added is generated by the photographing apparatus 100A.

  Hereinafter, the input profile information added to the multiband image data by the imaging device 100A is referred to as primary input profile information, and the input profile information added to the multiband image data by another information processing device different from the imaging device 100A is two. This is called next input profile information. The input profile information 700 described with reference to FIG. 7A in the first embodiment corresponds to the secondary input profile information in the second embodiment.

  The camera ID and the lens ID included in the primary input profile information 700A include information that can uniquely identify the imaging device 100A and the imaging lens 110A, respectively. By examining the primary input profile information 700A, it is possible to specify which model and which individual image input device has obtained the multiband image data to which this information is added.

  Original image data generated by the photographing apparatus 100A is stored in a first information processing apparatus as a content server. Hereinafter, the first information processing apparatus 1110 is referred to as a server 1110. In FIG. 11, each of the photographing apparatus 100 </ b> A and the server 1110 is illustrated one by one, but each of them may be plural or one.

  The transfer of the original image data from the photographing apparatus 100A to the server 1110 can be performed through various methods such as a memory card, wired communication, wireless communication, and a network. For example, when the photographing apparatus 100A is provided with a network connection function via a wireless LAN, a wired LAN, or the like, the original image data can be transferred to the server 1110 via the WAN / LAN 1100. It is also possible to connect the memory card taken out from the photographing apparatus 100A to a computer or the like and temporarily store it in the computer. The computer can be connected to the server 1110 via the WAN / LAN 100 and the original image data stored in the computer can be transferred to the server 1110. Hereinafter, the WAN / LAN 1100 is simply referred to as a network 1100.

  The server 1110 can be provided and operated by a manufacturer of an image input device such as the imaging device 100A or the imaging lens 110A. The server 1110 also accumulates a variety of information related to the color reproduction characteristics of image input devices such as the imaging device 100A and the imaging lens 110A. Then, the image input device ID information is read out from the input profile information added to the original image data generated by the photographing apparatus 100A, and related to the color reproduction characteristics of the image input device corresponding to the image input device ID information. It is possible to acquire various information.

  That is, various pieces of information related to the color reproduction characteristics of the image input device are stored in the memory 205 inside the photographing apparatus 100 in the first embodiment, whereas in the second embodiment, the server 1110 is stored. Is accumulated within.

  The server 1110 is configured to be connectable to the second information processing apparatus 140A as a client information processing apparatus via the network 1100. Hereinafter, the second information processing apparatus 140A is referred to as a client 140A. There may be one client 140A or a plurality of clients.

  In the following description, the client 140A is assumed to be a computer to which an image output device such as the display device 144 and the printer 150 is connected. However, the present invention is an image reproducing device, a video recorder, a set top box (STB), a printer, an image The present invention can also be applied to a portable data storage device with a display function, a mobile phone, and the like.

  The difference between the imaging apparatus 100 of the first embodiment and the imaging apparatus 100A of the second embodiment is that the input profile information added to the multiband image data generated by the imaging apparatus 100A is the primary input described above. It is only a point that is profile information. Other than that, the internal configuration of the image capturing apparatus 100A is the same as that described with reference to FIG. 2 in the first embodiment, so that the description thereof will be omitted, and the description will focus on differences from the image capturing apparatus 100. To do.

  FIG. 12 is a block diagram illustrating a schematic configuration of the supplementary information addition processing unit 600A included in the photographing apparatus 100A together with components connected to the supplementary information addition processing unit 600A. In FIG. 12, the same components as those of the photographing apparatus 100 according to the first embodiment shown in FIG.

  The imaging apparatus 100A has a body ID information storage unit 1210 instead of the sensor information storage unit 630 and the filter information storage unit 632 that the imaging apparatus 100 of the first embodiment has. The auxiliary information addition processing unit 600A includes a lens information acquisition unit 602A, a shooting information acquisition unit 604, a subject characteristic information input unit 606, a shooting illumination light information input unit 608, an input profile information generation unit 622A, and a data format process. Part 624.

  The lens information acquisition unit 602A acquires lens information including lens ID information and the like from the photographing lens 110A attached to the photographing apparatus 100A. The input profile information generation unit 622A inputs information described below from the body ID information storage unit 1210, the lens information acquisition unit 602A, the imaging information acquisition unit 604, the subject characteristic information input unit 606, and the imaging illumination light information input unit 608. The primary input profile information 700A shown in FIG. 13 is generated.

  Since the information sent from the imaging information acquisition unit 604, the subject characteristic information input unit 606, and the imaging illumination light information input unit 608 to the input profile information generation unit 622A is the same as that described in the first embodiment, the description thereof will be given. Omitted. Body ID information is sent from body ID information storage section 1210. The body ID information includes information that can identify the model of the photographing apparatus 100A and information that can identify an individual of the photographing apparatus 100A. The information that can identify the individual model of the photographing apparatus 100A can be text data representing the model name, product code, and the like. Information that can identify the individual of the imaging apparatus 100A can be information including a serial number.

  Lens ID information is sent from the lens information acquisition unit 602A. This lens ID information includes information that can identify the model of the photographing lens 110A and information that can identify the individual photographing lens 110A. The lens ID information also includes information similar to that described for the body ID information.

  When photographing is performed by the photographing apparatus 100A, the input profile information generation unit 622A generates primary input profile information 700A as illustrated in FIG. 13 and outputs the primary input profile information 700A to the data format processing unit 624. The primary input profile information 700A is added to the multiband image data output from the image processing unit 206, and processing for generating original image data is performed by the data format processing unit 624. This original image data is recorded in the storage medium 240. Is done.

  FIG. 13 is a diagram conceptually illustrating an example of the content of primary input profile information 700A generated by the input profile information generation unit 622. As illustrated in FIG. 13, the primary input profile information 700A includes lens ID information, body ID information, a focal length, an aperture value, and a reciprocal of the shooting distance set by the shooting lens 110A and the shooting apparatus 100A during shooting. It is possible to include shooting setting information, subject characteristic information, shooting illumination information, etc., including shutter speed, set ISO sensitivity, and the like. The subject characteristic information and the photographing illumination information are the same as those described in the first embodiment.

  14A and 14B show the supplementary information addition processing unit 1470 in the server 1110 and the components connected to the supplementary information addition processing unit 1470 (image data acquisition unit 1402, auxiliary storage device 1420, color conversion processing unit 1430). It is a schematic block diagram which shows. This supplementary information addition processing unit 1470 is provided in the server 1110. While the incidental information addition processing unit 600A in the image capturing apparatus 100A performs processing for adding primary input profile information to multiband image data, the incidental information addition processing unit 1470 receives secondary input profile information as described below. Processing to add to multiband image data is performed.

  The image data acquisition unit 1402, the color conversion processing unit 1430, and the incidental information addition processing unit 1470 may be implemented by dedicated hardware logic, or may be implemented by interpreting and executing a program stored in the server 1110. Good. In the following description, it is assumed that the image data acquisition unit 1402, the color conversion processing unit 1430, and the incidental information addition processing unit 1470 are executed by interpreting and executing a program stored in the server 1110. The auxiliary storage device 1420 is configured by a hard disk drive or the like built in or externally attached to the server 1110.

  The incidental information addition processing unit 1470 includes an input device ID information acquisition unit 1404, a shooting setting information acquisition unit 1406, a shooting illumination light information acquisition unit 1408, a subject characteristic information acquisition unit 1410, a body ID information acquisition unit 1412, It has a lens ID information acquisition unit 1414, an input profile generation information acquisition unit 1416, and a bias / tone curve information acquisition unit 1418 (hereinafter, FIG. 14A). The incidental information addition processing unit 1470 further includes an incident angle derivation unit 612S, a sensitivity unevenness derivation unit 614S, a filter information acquisition unit 1434, a peripheral light attenuation derivation unit 610S, an input profile information generation unit 622S, and a data format processing unit. 624S (above, FIG. 14B).

  The input profile information generation unit 622S includes a filter characteristic change amount derivation unit 618S, a region division processing unit 620S, an overall spectral sensitivity characteristic derivation unit 626S, a representative characteristic information generation unit 628AS, a correction coefficient generation unit 628BS, and an information aggregation process. Part 628CS.

  The incidental information addition processing unit 1470 generates secondary input profile information based on the primary input profile information added to the multiband image data generated by the imaging apparatus 100A. Then, the secondary input profile information is added in place of the primary input profile information added to the multiband image data. The input profile information generation unit 622S has the same function as the input profile information generation unit 622 included in the imaging device 100 according to the first embodiment, and generates the secondary input profile information.

  Among the constituent elements described above, constituent elements having the same functions as those of the photographing apparatus 100 according to the first embodiment have characters at the end of the same reference numerals as those attached to the constituent elements of the photographing apparatus 100. The detailed description is omitted by adding S. For example, the area division processing unit 620 included in the imaging apparatus 100 in the first embodiment and the area division processing unit 620S included in the supplementary information addition processing unit 1470 of the server 1110 in the second embodiment are basically the same. Have the same function. The reason for adding the letter S to the end of the reference sign is to indicate that these components are provided in the server 1110 and not in the photographing apparatus 100A.

  The image data acquisition unit 1402 reads the original image data with primary input profile information corresponding to the image requested to be transmitted from the client 140A from the auxiliary storage device 1420 and transmits it to the additional information addition processing unit 1470.

  From the primary input profile information separated from the original image data sent from the image data acquisition unit 1402 to the incidental information addition processing unit 1470, the input device ID information acquisition unit 1404, the imaging setting information acquisition unit 1406, and the imaging illumination light information acquisition unit In 1408, the subject characteristic information acquisition unit 1410 acquires information to be described later. Multiband image data from which primary input profile information is separated (hereinafter simply referred to as “image data”) is output to the color conversion processing unit 1430 or the data format processing unit 624S.

  Whether the image data is output to the color conversion processing unit 1430 or the data format processing unit 624S depends on the type of image requested from the client 140A. That is, when the client 140A requests that the image data that has been subjected to the color conversion processing by the server 1110 be sent, the color conversion processing unit 1430 performs the color conversion processing and sends it to the client 140A. In this case, the client 140A outputs the image that has undergone the color conversion processing by the server 1110 as it is to an image output device such as the display device 144 or the printer 150. In this specification, image data that can be output to the image output apparatus without performing color conversion processing on the client 140A side that has received the image data is referred to as “output image data”.

  The input device ID information acquisition unit 1404 extracts the input device ID information from the primary input profile information separated from the original image data, and sends it to the body ID information acquisition unit 1412 and the lens ID information acquisition unit 1414.

  The shooting setting information acquisition unit 1406 extracts shooting setting information included in the primary input profile information, and sends it to the input profile generation information acquisition unit 1416 and the information aggregation processing unit 628CS. The shooting setting information is as described with reference to FIG.

  The photographic illumination light information acquisition unit 1408 extracts the photographic illumination information included in the primary input profile information and sends it to the information aggregation processing unit 628CS. The subject characteristic information acquisition unit 1410 extracts subject characteristic information included in the primary input profile information, and sends it to the information aggregation processing unit 628CS.

  The body ID information acquisition unit 1412 sends the body ID information extracted from the primary input profile information to the input profile generation information acquisition unit 1416. The lens ID information acquisition unit 1414 sends lens ID information extracted from the primary input profile information to the input profile generation information acquisition unit 1416.

  The input profile generation information acquisition unit 1416 reads, from the auxiliary storage device 1420, input device information corresponding to the combination of information transmitted from the shooting setting information acquisition unit 1406, body ID information acquisition unit 1412, and lens ID information acquisition unit 1414. . Then, the read input device information is sent to the bias / tone curve information acquisition unit 1418, the incident angle derivation unit 612S, the filter information acquisition unit 1434, the sensitivity unevenness derivation unit 614S, and the peripheral light attenuation derivation unit 610S.

  By the way, the auxiliary storage device 1420 includes input device ID information that can uniquely identify various input devices including the photographing device 100A, the photographing lens 110A, and the like, and sensor information, lens information corresponding to the input device ID information, Filter information and bias / tone curve information (these information are the same as those described in the first embodiment, and are stored in the photographing apparatus 100 and the photographing lens 110 in the first embodiment. Is comprehensively accumulated).

  Information is provided by manufacturers of input devices such as the photographing apparatus 100A and the photographing lens 110A. It should be noted that the user who purchased the photographing apparatus 100A and the photographing lens 110A may calibrate by himself / herself or at a repair center or the like when a certain period of time has elapsed since the start of using the photographing apparatus 100A or the photographing lens 110A. . By sending the calibration result to the server 1110, sensor information, lens information, filter information, bias / tone curve information, and the like can be updated as appropriate.

  The bias / tone curve information acquisition unit 1418 acquires the bias / tone curve information from the information sent from the auxiliary storage device 1420 and sends it to the information aggregation processing unit 628CS.

  The incident angle deriving unit 612S performs processing similar to the processing in the incident angle deriving unit 612 in the first embodiment, and outputs the processing result to the input profile information generating unit 622S.

  The filter information acquisition unit 1434 acquires filter information (filter information) built in the imaging apparatus 100A from the input device information sent from the input profile generation information acquisition unit 1416. Then, the filter information acquisition unit 1434 outputs the filter information to the input profile information generation unit 622S. This filter information is the same as that stored in the filter information storage unit 632 of the photographing apparatus 100 in the first embodiment.

  The sensitivity unevenness deriving unit 614S extracts information related to the sensitivity unevenness of the imaging apparatus 100A from the input device information sent from the input profile generation information acquisition unit 1416, and outputs the information to the input profile information generation unit 622S. As in the case of the first embodiment, this sensitivity unevenness information is caused by at least one of the image sensor, the processing system that processes the image signal output from the image sensor, and the shutter. This is information reflecting the spectral sensitivity unevenness that occurs equivalently on the image area of the image sensor.

  The peripheral dimming derivation unit 610S extracts information related to the peripheral dimming characteristics of the photographing lens 110A from the input device information sent from the input profile generation information acquisition unit 1416, and the input profile information generation unit 622S (input profile) To the correction coefficient generation unit 628BS) in the information generation unit 622S. The input profile information generation unit 622S performs the same processing as the input profile information generation unit 622 described with reference to FIG. 6B in the first embodiment to generate secondary input profile information, and the data format processing unit 624S. And output to the color conversion processing unit 1430.

  When the output form of the image data requested from the client 140A is output image data, the color conversion processing unit 1430 performs color conversion processing on the image data to generate output image data. That is, the color conversion processing unit 1430 uses the secondary input profile information generated by the information aggregation processing unit 628CS in the input profile information generation unit 622S and the output profile information sent from the client 140A to generate image data. Color conversion processing is performed to generate output image data.

  On the other hand, when image data transmission with secondary input profile information is requested from the client 140A, the input profile information generation unit 622S generates secondary input profile information. As described above, the secondary input profile information is the same information as described with reference to FIG. 7A in the first embodiment. The input profile information generation unit 622S generates secondary input profile information through processing similar to the processing in the input profile information generation unit 622 included in the imaging apparatus 100 according to the first embodiment.

  Incidentally, the client 140A may request the server 1110 to transmit a plurality of images generated by the same input device. At that time, the photographing lens 110A may be a zoom lens, and the plurality of images may be captured and generated with different set focal lengths. In such a case, the set of information including the divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region described with reference to FIG. A plurality of lenses 110A can be prepared corresponding to the focal lengths that can be set by the zoom lens.

  That is, as described with reference to FIG. 7D in the first embodiment, sets of information 702-1 and 702-2 including divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each region. ,... 702-n can be generated in advance. Here, “pre-generation” means that when there is a request from the client 140A to the server 1110 to send out a plurality of image data generated by the combination of the same photographing device 100A and photographing lens 110A, the server 1110 This means that the set of information is generated before sending a series of image data to the client 140A.

  When the client 140A is configured to be able to perform color conversion processing based on the secondary input profile information added to the image data sent from the server 1110, the client 140A includes a color conversion processing unit. Have. This color conversion processing unit can be the same as that described with reference to FIG. 9 in the first embodiment.

  FIG. 15 is a schematic flowchart illustrating a procedure of multiband image data acquisition processing executed by the client 140A. A program stored in the storage device in the client 140A is read out, and the processing device in the client 140A executes the program to perform multiband image data acquisition processing.

  In S1502, the client 140A receives an input of the type of image to be acquired. That is, a setting operation performed by the user of the client 140A is accepted depending on whether image data for output is desired as an image transmitted from the server 1110 or image data with secondary input profile information is desired.

  In S <b> 1504, the client 140 </ b> A determines whether the operator desires to acquire output image data based on the operator's setting operation received in S <b> 1502. If the determination in step S1504 is affirmed, the client 140A requests the server 1110 to send output image data in step S1506.

  In S1508, the client 140A outputs the output profile to the server 1110. This output profile includes the output profile of the image output apparatus included in or connected to the client 140A, information on the characteristics of illumination light (referred to as observation illumination light) that illuminates the environment where the client 140A exists, and the like. Can be included. In step S1510, the client 140A receives the output image data sent from the server 1110.

  If the determination in S1504 is negative, that is, if it is determined that the operator wants to acquire image data with secondary input profile information, the process proceeds to S1514. In S1514, the client 140A requests the server 1110 to transmit image data with secondary input profile information.

  In S1516, the client 140A receives image data with secondary input profile information from the server 1110. In step S1518, the client 140A performs color conversion processing using the secondary input profile information added to the image data transmitted from the server 1110 and the output profile information stored in the client 140A. Regarding the color conversion processing in S1518, it is possible to perform the same processing as that described with reference to FIGS. 10A and 10B in the first embodiment.

  In step S1512, the client 140A transmits the output image data received in step S1510 or the image data generated by the color conversion processing in step S1518 to an image output device such as the display device 144 or the printer 150, and performs multiband image data acquisition processing. Complete.

  FIG. 16 is a schematic flowchart illustrating a procedure of multiband image data transmission processing executed by the server 1110. A program stored in the storage device in the server 1110 is read out and executed by the server 1110, whereby multiband image data transmission processing is performed.

  In step S1602, the server 1110 receives the image transmission request issued from the client 140A. In step S1604, the server 1110 determines whether the image data requested to be transmitted from the client 140A has secondary input profile information. If the determination in S1604 is affirmative, that is, if it is determined that the image data requested to be transmitted from the client 140A has secondary input profile information, the process proceeds to S1606.

  On the other hand, if it is determined in S1604 that the image data requested to be transmitted from the client 140A does not have the secondary input profile information, the secondary input profile information is generated and added to the image data in S1620. The processing proceeds to S1606. The process in S1620 can be the same as the process described with reference to FIGS. 8A and 8B in the first embodiment.

  Here, the processing in S1604 will be described. The image data requested to be transmitted from the client 140A may have been requested to be transmitted as image data with secondary input profile information from another client in the past. In such a case, the secondary input profile information generated in the past can be stored together with the image data. Alternatively, when the photographing apparatus 100A has the same specifications as the photographing apparatus 100 of the first embodiment, the image data is stored in the secondary input profile when the original image data is stored in the auxiliary storage device 1420 of the server 1110. Information is attached. As described above, in the case of image data in which secondary input profile information has been generated in the past, or in the case of image data generated by a photographing apparatus capable of generating input profile information as shown in FIG. 7A, in S1604. This determination is affirmed.

  In step S1606, the server 1110 determines whether the type of image data requested from the client 140A is output image data. If the determination in S1606 is affirmative, that is, if the type of image data requested from the client 140A is output image data, the process advances to S1608. In step S1608, the server 1110 receives the output profile information from the client 140A.

  In step S1610, the server 1110 performs color conversion processing. This color conversion processing can be performed by the method described with reference to FIGS. 10A and 10B in the first embodiment. In this case, the secondary input profile information added to the image data (or generated in S1620) is used in the color conversion process. That is, an input profile is created using the spectral reflectance estimation matrix for each region, and matrix calculation is performed for each region.

  In step S1612, the server 1110 transmits image data to the client 140A, and completes a series of multiband image data transmission processing.

  In the above description with reference to the flowchart of FIG. 16, the secondary input profile information is generated when it is determined that the image data requested to be transmitted from the client 140A does not have the secondary input profile information. It was a thing. In this regard, it is also possible to do the following. That is, when the original image data generated by the image input device is accumulated in the auxiliary storage device 1420, it can be determined whether or not secondary input profile information is added to the original image data. Then, when it is determined that the secondary input profile information is not added, it is possible to perform a process of generating the secondary input profile information.

  At this time, for example, there may be a case where a plurality of original image data generated by the combination of the imaging device 100A and the imaging lens 110A is stored in the auxiliary storage device 1420. In such a case, a set of information 702 including divided area representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each area as described with reference to FIG. 7D in the first embodiment. 1, 702-2,..., 702-n can be generated in advance. “Preliminarily generating” means that the above information is generated in advance before generating the secondary input profile information for each of a plurality of series of image data captured and generated by the same combination of the imaging device 100A and the imaging lens 110A. To generate a set of In this way, it is possible to improve the efficiency of the secondary input profile information generation process.

  The second embodiment of the present invention has the following advantages. First, the primary input profile information generated by the imaging apparatus 100A has a smaller amount of information than the secondary input profile information, and the processing load when generating the primary input profile information is reduced. Therefore, it is possible to increase the continuous shooting speed (frame speed) when continuous shooting is performed with the shooting apparatus 100A. Further, more original image data can be recorded on the storage medium 240.

  Furthermore, when image data with an input profile is sent from the server 1110 to the client 140A, the amount of secondary input profile information added to the image data is added to the input profile information corresponding to each pixel. It is possible to greatly reduce as compared with the case of. Accordingly, it is possible to shorten the time required for transferring the image data.

  The technology of multiband images according to the present invention includes a digital still camera, a digital movie camera, a photographing device such as a camera incorporated in a mobile phone, a server for content transmission, a set top box used by being connected to a monitor display device, The present invention can be applied to a video player, a video recorder, and the like.

100, 100A Imaging device 102 Camera CPU
110, 110A Photography lens 112 Lens CPU
140 Information processing device 140A Second information processing device (client)
142 PC
144 Display device 150 Printer 206 Image processing unit 210, 212 Shutter 214, 216 Optical filter 218, 220 Image sensor 240 Storage medium 600, 600A, 1470 Attached information addition processing unit 602 Lens information acquisition unit 604 Shooting information acquisition unit 606 Subject characteristic information Input unit 608 Imaging illumination light information input unit 610, 610S Peripheral dimming deriving unit 612, 612S Incident angle deriving unit 614, 614S Sensitivity unevenness deriving unit 616 Bias / tone characteristic extracting unit 618, 618S Filter characteristic change amount deriving unit 620, 620S Region division processing unit 622, 622A, 622S Input profile information generation unit 624, 624S Data format processing unit 626, 626S Total spectral characteristic derivation unit 628A, 628AS Representative characteristic information generation unit 628B, 628BS Correction coefficient Generation unit 628C, 628CS Information aggregation processing unit 700 Input profile information 700A Primary input profile information 702-1, 702-2 Set of information 900 Color conversion processing unit 910 Input profile processing unit 912 Tone correction unit 914 Area division unit 916 For each area Matrix operation unit 920 Output profile operation unit 930 Input profile creation unit 932 Bias / tone curve correction information acquisition unit 934 Region division information acquisition unit 936 Spectral reflectivity estimation matrix creation unit for each region 938 System function calculation unit for each region 940 For each region Spectral Sensitivity Creation Unit 942 Imaging Illumination Information Acquisition Unit 950 Image Output Device 1110 First Information Processing Device (Content Server)
1402 Image data acquisition unit 1404 Input device ID information acquisition unit 1406 Shooting setting information acquisition unit 1408 Shooting illumination light information acquisition unit 1410 Subject characteristic information acquisition unit 1412 Body ID information acquisition unit 1414 Lens ID information acquisition unit 1416 Input profile generation information acquisition Unit 1418 Bias / tone curve information acquisition unit 1420 Auxiliary storage device 1430 Color conversion processing unit

Claims (15)

An imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. An imaging device capable of generating multiband image data of a number of bands larger than the number of colors,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. A first spectral characteristic distribution information generation unit that generates characteristic distribution information;
A second spectral characteristic distribution information generation unit that generates second spectral characteristic distribution information that reflects non-uniformity of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. An input profile information generator to
A data format processing unit that performs format processing by adding the input profile information to the multiband image data;
The input profile information generation unit
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area A representative characteristic information generating unit that generates divided area representative characteristic information that is information;
An imaging apparatus, comprising: an information aggregation processing unit that aggregates the divided region representative characteristic information together with other information to generate the input profile information. The representative characteristic information generation unit further includes:
A divided area representative characteristic information reduction process, which is a process of reducing the number of divided area representative characteristic information equal to the number of divided areas generated for each of the plurality of divided areas to a number smaller than the number of divided areas. Information for designating which divided area representative characteristic information to be applied among the divided area representative characteristic information reduced to a number smaller than the number of divided areas corresponding to each of the plurality of divided areas. The photographing apparatus according to claim 1, wherein the photographing apparatus is configured to generate representative characteristic designation information. The input profile information generation unit further includes
In order to correct the peripheral light reduction characteristics due to the peripheral light amount reduction characteristics of the photographing optical system and the image height dependence of the incident light quantity when the subject light is incident on each pixel in the image area of the image sensor, The imaging apparatus according to claim 2, further comprising a correction coefficient generation unit that generates a spectral sensitivity correction coefficient for each area corresponding to the plurality of divided areas. The photographing optical system is a variable magnification optical system that can be set to a plurality of focal lengths,
The input profile information generation unit further includes
Prior to photographing, a plurality of sets of divided region representative characteristic information, representative characteristic designation information, and spectral sensitivity correction coefficient for each region corresponding to each of the plurality of focal lengths that can be set by the photographing optical system are generated. The set corresponding to the focal length set by the photographing optical system at the time of photographing is selected from the plurality of sets of divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region. The imaging apparatus according to claim 3, wherein the imaging apparatus is configured to select the divided area representative characteristic information, the representative characteristic designation information, and a spectral sensitivity correction coefficient for each area. A color conversion processing device for performing color conversion processing on multiband image data generated by the photographing device according to claim 1,
Based on the divided region representative characteristic information, an input profile creating unit that creates an input profile for each divided region corresponding to each of the plurality of divided regions;
An input profile operation unit that uniformly applies an input profile created corresponding to the divided area to all pixel data in each divided area of the plurality of divided areas. Color conversion processing device. An imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. A server capable of storing multiband image data generated by a photographing device capable of generating multiband image data of a number of bands larger than the number of colors and sending the multiband image data to a client information processing device An information processing apparatus,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. A first spectral characteristic distribution information generation unit that generates characteristic distribution information;
A second spectral characteristic distribution information generation unit that generates second spectral characteristic distribution information that reflects non-uniformity of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. An input profile information generator to
A data format processing unit that performs format processing by adding the input profile information to the multiband image data;
The input profile information generation unit
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area A representative characteristic information generating unit that generates divided area representative characteristic information that is information;
A server information processing apparatus comprising: an information aggregation processing unit that aggregates the divided region representative characteristic information together with other information to generate the input profile information. The representative characteristic information generation unit further includes:
A divided area representative characteristic information reduction process, which is a process of reducing the number of divided area representative characteristic information equal to the number of divided areas generated for each of the plurality of divided areas to a number smaller than the number of divided areas. Information for designating which divided area representative characteristic information to be applied among the divided area representative characteristic information reduced to a number smaller than the number of divided areas corresponding to each of the plurality of divided areas. The server information processing apparatus according to claim 6, wherein the server information processing apparatus is configured to generate representative characteristic designation information. The input profile information generation unit further includes
In order to correct the peripheral light reduction characteristics due to the peripheral light amount reduction characteristics of the photographing optical system and the image height dependence of the incident light quantity when the subject light is incident on each pixel in the image area of the image sensor, The server information processing apparatus according to claim 7, further comprising a correction coefficient generation unit that generates a spectral sensitivity correction coefficient for each area corresponding to the plurality of divided areas. The photographing optical system is a variable magnification optical system that can be set to a plurality of focal lengths,
The input profile information generation unit further includes
A plurality of sets of divided region representative characteristic information corresponding to each of the plurality of focal lengths that can be set by the photographing optical system, the representative characteristic designation information, and a spectral sensitivity correction coefficient for each region are generated in advance, The divided region of the set corresponding to the focal length set by the photographing optical system at the time of photographing from among the plural regions of the divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region. 9. The server information processing apparatus according to claim 8, wherein the server information processing apparatus is configured to select representative characteristic information, the representative characteristic designation information, and a spectral sensitivity correction coefficient for each region. A client information processing apparatus capable of receiving the multiband image data transmitted from the server information processing apparatus according to any one of claims 6 to 9 and performing color conversion processing,
Based on the divided region representative characteristic information, an input profile creating unit that creates an input profile for each divided region corresponding to each of the plurality of divided regions;
An input profile operation unit that uniformly applies an input profile created corresponding to the divided area to all pixel data in each divided area of the plurality of divided areas. Client information processing device. An imaging device capable of generating color image signals of a plurality of colors corresponding to a subject image formed on an image area by a photographing optical system;
An optical filter disposed on either the front side, the middle side, or the rear side of the photographing optical system and capable of changing a spectral characteristic of subject light incident on the imaging device via the photographing optical system. Multi-band image data generated by a photographing device capable of generating multi-band image data of a number of bands larger than the number of colors is accumulated, and the multi-band image data can be transmitted to the client information processing device An image data processing method in a server information processing apparatus,
A first spectrum reflecting unevenness in spectral sensitivity equivalently occurring on the image area due to at least one of the image sensor, a processing system for processing an image signal output from the image sensor, and a shutter. Generating characteristic distribution information;
Generating second spectral characteristic distribution information reflecting unevenness of spectral characteristics generated in a subject image formed on the image area due to the photographing optical system and the optical filter;
Based on both the first and second spectral characteristic distribution information or only the second spectral distribution information, information representing the total spectral sensitivity characteristic of the image sensor is generated, and input profile information including the information is generated. To do
Adding the input profile information to the multiband image data and performing format processing;
Generating the input profile information;
The image area is divided into a plurality of divided areas, and based on the total spectral sensitivity characteristics corresponding to each of a plurality of pixels included in each divided area, the total spectral sensitivity characteristics representing each divided area Generating divided region representative characteristic information that is information;
An image data processing method in a server information processing apparatus, comprising: generating the input profile information by aggregating the divided region representative characteristic information together with other information. Generating the divided region representative characteristic information;
A divided area representative characteristic information reduction process, which is a process of reducing the number of divided area representative characteristic information equal to the number of divided areas generated for each of the plurality of divided areas to a number smaller than the number of divided areas. Information for designating which divided area representative characteristic information to be applied among the divided area representative characteristic information reduced to a number smaller than the number of divided areas corresponding to each of the plurality of divided areas. The image data processing method in the server information processing apparatus according to claim 11, further comprising: generating representative characteristic designation information that is Generating the input profile information further;
In order to correct the peripheral light reduction characteristics due to the peripheral light amount reduction characteristics of the photographing optical system and the image height dependence of the incident light quantity when the subject light is incident on each pixel in the image area of the image sensor, The image data processing method in the server information processing apparatus according to claim 12, further comprising generating a spectral sensitivity correction coefficient for each area corresponding to the plurality of divided areas. The photographing optical system is a variable magnification optical system that can be set to a plurality of focal lengths,
Generating the input profile information further;
A plurality of sets of divided region representative characteristic information corresponding to each of the plurality of focal lengths that can be set by the photographing optical system, the representative characteristic designation information, and a spectral sensitivity correction coefficient for each region are generated in advance, The divided region of the set corresponding to the focal length set by the photographing optical system at the time of photographing from among the plural regions of the divided region representative characteristic information, the representative characteristic designation information, and the spectral sensitivity correction coefficient for each region. The image data processing method in the server information processing apparatus according to claim 13, further comprising selecting representative characteristic information, the representative characteristic designation information, and a spectral sensitivity correction coefficient for each region. A color in a client information processing apparatus capable of receiving multiband image data processed by the image data processing method in the server information processing apparatus according to any one of claims 11 to 14 and transmitted from the server information processing apparatus. A conversion processing method,
Creating an input profile for each divided region corresponding to each of the plurality of divided regions based on the divided region representative characteristic information;
The client information characterized in that the input profile created corresponding to the divided area is uniformly applied to all the pixel data in each divided area of the plurality of divided areas. A color conversion processing method in a processing apparatus.

Images