JP5530785B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly to a single-plate color imaging apparatus having a light source measurement function and a color correction function.

  As a conventional technique, there is Patent Document 1 as an apparatus for measuring the spectral characteristics of a light source. Further, there is Patent Document 2 as an apparatus for measuring reflection spectral characteristics of a subject. Further, as a technique for performing color correction from the measured spectrum of illumination light and the spectral reflectance of a color patch, there is Patent Document 3 and the like. Further, in Patent Document 4 and Patent Document 5 and the like, an imaging system that performs color correction of an input image using spectral spectrum information of a subject is proposed.

JP 62-185128 A JP 2008-139062 A JP 2008-244613 A Japanese Patent No. 3614126 JP 2005-341175 A

  The present applicants have proposed Japanese Patent Application No. 2009-170722 (hereinafter referred to as “Application 1”) and Japanese Patent Application No. 2009-172605 (hereinafter referred to as “Application 2”) which have been proposed as devices for measuring the spectral characteristics of the light source of the subject. And the present application is a technique related to the application 1 in particular.

  Specifically, in Application 1, it is possible to measure the spectral characteristics of a captured image by setting a part of a single-plate color two-dimensional imaging device as a monochrome imaging region and applying appropriate optical filter processing to the monochrome imaging region. In Application 2, it is possible to measure the spectral characteristics of the captured image by guiding the captured image to the spectrometer with a mirror structure.

  However, in comparison with the form of a commonly used imaging device, in Application 1, it is necessary to perform special optical filter processing on the imaging element, and in Application 2, there is a problem that the imaging device becomes large with a special structure. there were.

  An object of the present invention is to realize an imaging apparatus capable of accurately performing color correction on an illumination light source of a subject being photographed without greatly changing the form of a commonly used imaging apparatus.

In the present invention for achieving the above object, to allow the spectral characteristics measurement of the photographic image by connecting the other end to place the light-receiving surface near the light guide fiber end of the image pickup device to the spectrometer, the cross section of the optical fiber lens optical Installed so that it faces the axis, or forms a light transmission diffusing material or reflection or light refraction direction changing material in front of or within the optical fiber, and the optical fiber is at an angle other than the lens optical axis Incident light from the lens is incident on the optical fiber .

  According to the present invention, it is possible to realize an imaging apparatus capable of accurately performing color correction on an illumination light source of a subject being photographed without greatly changing the form of a commonly used imaging apparatus. For example, it is possible to automatically perform an optimum color correction process according to the light source condition at an arbitrary shooting location and the optical characteristics of the lens or the like during white balance adjustment.

First embodiment of the present invention Example of arrangement of image pickup element and optical fiber viewed from the lens side in the embodiment of the present invention Example of optical fiber installation in an embodiment of the present invention Second embodiment of the present invention

  Embodiments of the present invention will be described below with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to what has the same function, and the overlapping description is abbreviate | omitted.

  An embodiment using the present invention will be described with reference to FIG.

  In FIG. 1, 101 is a subject, 102 is a lens, 103 is a lens image circle, 104 is an image sensor, 105 is an optical fiber, 106 is a spectroradiometer, 107 is a camera process circuit and color correction process circuit, and 108 is an optical fiber end. .

  Incident light from the lens 102 is converted from an optical signal to an electric signal by the image sensor 104 disposed at the image formation position, and is input as a video signal to the camera process circuit and the color correction process circuit 107 to perform appropriate video signal processing. And then output as a color-corrected video signal. An optical fiber 105 (light guide fiber or simply fiber), which is a fiber material for guiding light close to the imaging surface, is installed above or below the imaging element 104 in the vertical direction. At the other end of the optical fiber 105 (optical fiber end opposite to the image sensor side), a spectroradiometer 106 (spectral intensity meter or simply spectrometer) capable of measuring the intensity for each wavelength of light is installed. The signal output from the spectrometer 106 is input to the color correction process circuit of the camera process circuit and the color correction process circuit 107 and used for color correction signal processing.

  An arrangement example of the imaging element 104 and the optical fiber 105 viewed from the lens 102 side at this time is shown in FIG. A circle indicated by a dotted line in FIG. 2 indicates a lens image circle 103 that is a range in which light is imaged by the lens. At this time, since the lens 102 is circular, the lens image circle 103 is circular. On the other hand, the shape of the effective imaging area of the image sensor 104 is generally not a circle but a rectangle, and the diagonal of the image sensor 104 is designed to be within the lens image circle 103. Further, the rectangle has horizontal and vertical lengths. For example, the ratio is 3: 2 for the same ratio as 35mm film camera, 4: 3 for NTSC television camera, 16: 9 for HDTV television camera, etc. is there. Therefore, in most general image pickup apparatuses, there is a region that is normally imaged by a lens near the image pickup surface of the image pickup device in the vertical direction but is not used for a video signal. The optical fiber end 108 is installed in an arbitrary area not used for imaging in such an image circle.

  Next, an installation example of the optical fiber is shown in FIG. FIG. 3A shows an example in which the optical fiber faces the optical axis of the lens, FIG. 3B shows an example in which a light transmission diffuser (or a light diffusion plate or light diffusion film) is installed in front of the optical fiber, and FIG. A spherical light transmissive diffuser is formed, or a structure similar to that used for LED light diffusion caps and light diffusion LEDs is used, so that light from the side can be incident on the optical fiber. An example in which the optical fiber is installed at an angle other than the pair, (d) is an example in which a light diffusing fiber is used at the tip of the optical fiber, and the optical fiber is installed at an angle other than the front to the lens optical axis. By using a material that changes the direction of light refraction by scratching or shaving the part, or by using a material that changes the direction of light refraction, etc. An example in which the fiber is installed at an angle other than the right angle with respect to the optical axis of the lens. (F) is a structure in which a minute prism or mirror-like reflector is installed at the tip of the optical fiber and the light from the side of the fiber is incident on the optical fiber. This is an example in which the optical fiber is installed at an angle other than the front facing the lens optical axis.

  As described above, the optical fiber end 108 is installed in the lens image circle 103 near the imaging surface of the image sensor 104 and in an area not used for imaging. As an installation form, the optical fiber may be installed so that the cross section of the optical fiber faces the optical axis of the lens, or a light transmitting / diffusing material having an arbitrary shape (for example, plate shape, spherical shape) in front of the optical fiber or in the fiber. Alternatively, a reflection or photorefractive direction changing material may be formed, and the incident light from the lens may be incident on the optical fiber even when the optical fiber is at an angle other than the front of the lens.

  With the above-described structure, it is possible to measure the spectral intensity distribution of a part of incident light of a captured image at the same time as capturing.

  Next, the principle of performing color correction processing from the spectral distribution of the light source irradiated to the photographic subject is a technique based on color engineering, but will be briefly described here.

このとき、被写体の分光反射率ρ（λ）を標準白色板などを用いてなるべく平坦にする、もしくは可能であれば既知のデータで校正することにより、被写体からの反射光Ｃｓ（λ）は、光源の分光分布Ｐ（λ）をできるだけ反映した特性とすることができる。すなわち、Ｃｓ（λ）のスペクトルを測定することにより、撮影環境の光源の分光特性を推定することが可能となる。また、ＲＧＢセンサの分光撮像特性をあらかじめ測定しておけば、そのセンサ出力ＲＧＢ値を計算で求めることが可能となる。従って、例えばマクベスチャートなど反射分光特性ρ（λ）が既知のデータを用いると、光源の分光特性を測定するだけで、各照明条件におけるカメラで撮影したマクベスチャートの再現色を計算で作成することができる。 The subject of the camera is illuminated by a light source such as illumination or the sun, and the reflected light is incident on the image sensor through the lens of the camera. At this time, the illumination has a spectral distribution P (λ) having an intensity corresponding to the wavelength λ of light, which is determined by the light source. The subject also has a spectral reflectance ρ (λ). The product of both is incident on the TV camera as reflected light from the subject. The television camera receives the reflected light through an optical system transmission characteristic L (λ) such as a lens, an infrared (IR) cut filter, an optical low pass filter (LPF), or the like. Each pixel of the image is picked up by a color sensor having spectral sensitivities S _R (λ), S _G (λ), and S _B (λ), and outputs (RGB tristimulus values) corresponding to the integrated values are R, G, B Are output as color channel output signals. This is expressed as follows.

At this time, the reflected light Cs (λ) from the subject is obtained by flattening the spectral reflectance ρ (λ) of the subject as much as possible using a standard white plate or by calibrating with known data if possible. The characteristic can reflect the spectral distribution P (λ) of the light source as much as possible. That is, by measuring the spectrum of Cs (λ), it is possible to estimate the spectral characteristics of the light source in the shooting environment. If the spectral imaging characteristics of the RGB sensor are measured in advance, the sensor output RGB value can be obtained by calculation. Therefore, using data with known reflection spectral characteristics ρ (λ) such as a Macbeth chart, the reproduction color of the Macbeth chart photographed with the camera under each illumination condition can be created by calculation only by measuring the spectral characteristics of the light source. Can do.

In general, when shooting with a broadcast camera, in order to adjust the white balance of the camera according to the shooting environment, an achromatic reflection subject such as a gray or white standard reflection chart is shot before shooting, and the RGB signal intensity at that time To adjust the RGB amplifier gain settings. Therefore, first, an achromatic standard reflection chart is photographed to fill the screen, and if the spectral reflectance at that time is flat (ρ _white (λ) = constant), it is used as it is, or spectral reflectance ρ _white (λ ) Is known, the spectral characteristic P (λ) of the light source is calibrated according to the characteristic, and the spectral characteristic of light incident on the image sensor via the spectral characteristic L (λ) of the optical system is Obtained by spectrometer. Next, the reflection spectral characteristic ρ _{macbeth_color_k} (λ) (k is a color number 1 to 24) of a 24-color Macbeth chart whose reflection spectral characteristic is known, and the color spectral sensitivity characteristic S _R1 (λ) of a single-plate color camera measured in advance. , S _G1 (λ), S _B1 (λ), and color spectral sensitivity characteristics S _R2 (λ), S _G2 (λ), S _B2 (λ) of, for example, a three-plate color camera for broadcasting, which is the target of color correction, By accumulating, it is possible to generate by calculation the RGB signal amount of each color when it is assumed that a 24-color Macbeth chart has been shot with each of the single-plate camera and the three-plate camera.

  For example, color correction can be performed using RGB signals corresponding to Macbeth chart images taken by two types of cameras obtained by this calculation. For example, when color correction is performed using a linear matrix, the RGB values of each of the 24 colors of the Macbeth chart calculated assuming a single-plate camera are closest to the RGB values of each of the 24 colors of the Macbeth chart calculated assuming a 3-plate camera. Further, the linear matrix correction coefficient is obtained by the least square method, and the obtained linear matrix coefficient is subjected to the linear matrix color correction processing in the color correction processing circuit of the single-plate camera, so that the color reproduction characteristics of the single-plate camera can be reduced to 3 It is possible to approximate the color reproduction characteristics of a plate camera.

  Note that the assumed color chart (color chart) is not limited to the 24-color Macbeth chart if the reflection spectral characteristics are known, and any other color chart may be used. For example, the radiation spectral characteristics are known. An arbitrary color chart may be enlarged and photographed using an illumination light source and measured and calibrated with a spectrometer installed in this embodiment, and the reflection spectral characteristics of the color chart may be measured and used. Further, the color correction process need not be limited to the linear matrix method, and any adjustment method based on the obtained RGB signal information, such as gain / pedestal adjustment, multi-axis hue adjustment, look-up table method, painting process, etc. Color correction may be performed with

  That is, the color correction process circuit of the camera process circuit and the color correction process circuit 107 obtains the spectral characteristics of the light source when photographing an achromatic reflection subject such as a gray or white standard reflection chart, and the known reflection spectral characteristics. And the color spectral sensitivity characteristics of a single-plate color camera using a single-plate color image sensor that has been measured in advance and the color spectral sensitivity characteristics of a three-plate color camera as a color correction target. A photographed image is generated by calculation, and color correction processing is performed to reduce the color difference between the two types of images obtained by this calculation. Alternatively, the color correction process circuit estimates the light source from the characteristics of the spectral characteristics of the light source when an achromatic reflection subject such as a gray or white standard reflection chart is photographed, and uses the correction factor of the light source already recorded. An estimated light source correction coefficient may be selected, and color correction processing may be performed using the selected correction coefficient.

  Further, as shown in Application 1, the measurement or adjustment may be performed simultaneously with the white balance adjustment or simultaneously with the start of the color correction adjustment in conjunction with a white balance adjustment start switch or a color correction adjustment start switch (not shown). In addition, a lens defocus circuit (not shown) is provided, and in conjunction with the white balance adjustment start switch or the color correction adjustment start switch, the focus position of the lens 102 is set to MOD (shortest shooting distance, closest distance) or infinity when the color correction adjustment starts. Thus, the incident light to the optical fiber end 108 can be made as diffuse light as much as possible, and the influence of unevenness in brightness on the surface of the achromatic color chart of the subject can be suppressed, so that the spectral characteristic measurement can be performed more accurately. .

  At this time, the measurement result is recorded in the memory circuit in conjunction with or not in conjunction with the measurement of the spectral characteristics, and can be arbitrarily read out, so that any measurement according to the data measured in advance can be performed without performing the measurement on the spot. Color correction processing can be performed. For example, by measuring and recording the spectral characteristics at the time of the A light source and the D65 light source, or recording data measured in advance in a read-only memory (ROM), and arbitrarily selecting and reading as a preset value at the time of shooting For example, even when measurement is not possible before or during shooting, it is possible to perform color correction with values corresponding to the A light source and D65 light source in a fixed manner.

  Even when the measurement result of the spectral characteristic is recorded in the memory circuit, the data to be recorded is only the spectral characteristic. For example, even if the range of 380 to 750 nm is stored as 16-bit data every 5 nm, the amount is about 150 bytes. In addition, correction data can be indirectly recorded with a much smaller capacity than storing all correction data.

  In the present embodiment, the optical fiber end 108 is disposed in the lens image circle 103 near the imaging surface of the image sensor 104 and in an area not used for photographing. Generally, the optical fiber end 108 is interposed between the image pickup lens and the image sensor. What is necessary is just to provide.

  The image sensor 104 described above is most preferably a two-dimensional image sensor for single plate color that can acquire a color signal with one image sensor by performing on-chip color filter processing on the surface of the effective area of the image sensor. A two-dimensional image sensor may be used. Further, an optical fiber end may be provided between the imaging lens and the imaging element in an imaging apparatus that captures a color signal as a moving image or a still image using an optical spectroscopic prism and a plurality of monochrome imaging elements.

  A second embodiment using the present invention will be described with reference to FIG.

  In FIG. 4, the spectral characteristic data is added to the image data as metadata and output from the camera, and recorded in the external recording device 109. For example, an ancillary area or an audio data area of the HD-SDI signal is used to be superimposed and recorded on the image signal. Alternatively, the spectral characteristic data is output from the camera using metadata and other means than the image data, and is recorded separately from the image data. For example, recording is performed in association with an image signal using a memory unit of a VTR with memory.

  Thus, by recording spectral characteristic data at the time of photographing together with a video signal, for example, it is possible to perform optimum color correction processing at the time of reproduction or editing without performing color correction at the time of photographing.

  Except for the above points, the second embodiment is the same as the first embodiment.

  As mentioned above, the invention made by the present inventor has been specifically described based on the above embodiments. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Of course.

DESCRIPTION OF SYMBOLS 101 ... Object, 102 ... Lens, 103 ... Lens image circle, 104 ... Image sensor, 105 ... Optical fiber, 106 ... Spectral radiometer, 107 ... Camera process circuit and color correction process circuit, 108 ... Optical fiber end, 109 ... External recording device

Claims (6)

By applying on-chip color filter processing on the surface of the effective area of the image pickup device, a color signal can be obtained using a single-plate color image pickup device or an optical spectral prism and a plurality of monochrome image pickup devices. In an imaging device that captures moving images or still images,
Installing one end of an optical fiber capable of guiding light between the imaging lens and the imaging element in a region that does not use the imaging plane close to the lens image circle in a and photographing of the image pickup device, by connecting the other end to a spectrometer , Measure the spectral intensity distribution of the captured image ,
Install the optical fiber so that the cross-section of the optical fiber faces the optical axis of the lens, or construct an optically diffusing or reflecting or refraction-direction-changing material in front of the optical fiber or in the optical fiber so that the optical fiber is connected to the optical axis of the lens. Make sure that the incident light from the lens is incident on the optical fiber at an angle other than the front,
An imaging apparatus characterized by the above. The imaging device according to claim 1,
An image pickup apparatus using a single-plate color two-dimensional image pickup device capable of acquiring a color signal with a single image pickup device by performing on-chip color filter processing on a surface of an effective area of the image pickup device. The imaging device according to claim 1 or 2 ,
We obtain the spectral characteristics of the light source upon shooting achromatic reflective object, the known reflection spectral characteristics, and the color spectral sensitivity characteristic and color correction target of single-chip color camera using the single-chip color image pickup device previously measured So as to integrate the color spectral sensitivity characteristics of the three-plate color camera, generate an image photographed by the single-plate color camera and the three-plate color camera, and reduce the color difference between the two images obtained by this calculation. Correct color correction processing,
Alternatively, the light source is estimated from the characteristics of the spectral characteristics of the light source when the achromatic reflection subject is photographed, the estimated light source correction coefficient is selected from the already recorded light source correction coefficients, and the selected correction coefficient is selected. Using the color correction process,
An imaging apparatus characterized by the above. The imaging device according to claim 3 .
The operation of the color correction process circuit that performs the color correction processing is linked to the white balance adjustment start button or the color correction adjustment start button, and the color correction processing is performed simultaneously with the white balance adjustment or simultaneously with the start of the color correction adjustment. An imaging device that is characterized. In the imaging device according to any one of claims 1 to 4 ,
An imaging apparatus characterized in that a measurement result is recorded in a memory circuit and can be arbitrarily read out in conjunction or non-linkage with measurement of spectral characteristics. The imaging apparatus according to claim 5 ,
Measurement results recorded in conjunction or non-interaction with the measurement of spectral characteristics are added to the image data as metadata and output from the camera, or output from the camera using a means different from the image data, and stored in an external recording device. An imaging apparatus that is recorded.

Images