JP2006303783A - Image processing method, image processing system, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method, an image processing system, and an image processing program capable of acquiring a colorimetrical color reproduction image such as an XYZ image with high color reproducibility from a color image such as an RGB image. <P>SOLUTION: The image processing method images a white color board and a color chart area-sequentially to acquire the RGB image, and normalizes each RGB value of the white color board and the color chart by using the RGB value corresponding to the white color board. The method calculates a conversion parameter to convert the normalized RGB value of the white color board and the color chart on the basis of the normalized RGB value of the white color board and the color chart and an XYZ value measured by each color chart into an XYZ value. Succeedingly, the method images an optional object including the white color board area-sequentially to acquire a color image in the three primary colors, and normalizes each color image of the object by using the RGB value corresponding to the white color board. The method converts the RGB image of the object into the XYZ image on the basis of each normalized color image of the object and the calculated conversion parameter. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing system, and an image processing program, and more particularly to a technique for acquiring a color system image that does not depend on a device from color images of three primary colors captured in frame sequential order.

  Generally, a 3 × 3 matrix or a 3D-LUT (3D look-up table) is used to convert an image of three primary colors obtained through a color filter of three primary colors such as RGB into an XYZ image of the XYZ color system. (See Patent Documents 1 and 2).

Further, as a method of obtaining a color image through the three primary color filters, there is a frame sequential imaging method in which images of the three primary colors are acquired in a frame sequential manner. This type of field sequential imaging method rotates a rotating color filter having three primary color filters, and each time the color filter moves on the optical axis of the photographing optical system, the subject is passed through the color filter. Images are picked up by a black and white image pickup device, and images corresponding to the respective colors are sequentially captured.
JP 2001-94800 A JP-A-9-37091

  The frame sequential imaging method is a single-plate solid-state image sensor in which three primary color filters are arranged in a predetermined arrangement structure (Bayer, G-stripe, etc.), and uses a solid-state image sensor having the same number of pixels and size. Compared with an imaging apparatus, a high-resolution image can be obtained, and there is an advantage that the degree of freedom in designing a color filter is large.

  On the other hand, the frame-sequential imaging method is easily affected by variations in shutter speeds, fluctuations in the photographic light source, and the like at the time of photographing the three primary color images, and as a result, there is a possibility that an accurate three primary color image of the subject cannot be obtained. In addition, the position of the subject image on the image plane is shifted according to the rotational position of each rotary color filter due to the rotational axis of the rotary color filter, the mounting accuracy of the color filter, etc. Occurs).

  The present invention has been made in view of such circumstances, and is an image that can solve the above-described problems of the frame-sequential imaging method and can acquire a colorimetric color reproduction image such as an XYZ image with high color reproducibility. An object is to provide a processing method, an image processing system, and an image processing program.

  In order to achieve the above object, an invention according to claim 1 is an image processing method for acquiring a colorimetric color reproduction image from three primary color images captured in a frame sequential manner, and an exposure amount correction plate and a colorimetric conversion plate. A process of obtaining color images of three primary colors by imaging color charts in a surface sequential manner, and standardizing the color image values of the exposure amount correction plate and the color measurement conversion color sample with the corresponding color image values of the exposure amount correction plate, respectively. A first normalization step to be normalized, a color image value of the normalized exposure correction plate and colorimetric conversion color chart, and a colorimetric value measured for the exposure correction plate and colorimetric conversion color chart A step of calculating a conversion parameter for converting the standardized exposure amount correction plate and color image values of the colorimetric conversion color chart into the colorimetric values, and subjecting the subject including the exposure amount correction plate in a surface sequential manner. And acquiring the three primary color images, and acquiring each color image of the subject. A second normalization step of normalizing with a corresponding color image value of the colorimetric conversion color chart, the subject color image based on the standardized color image of the subject and the calculated conversion parameter; Converting to a colorimetric color reproduction image.

  First, conversion parameters for converting a color image of the three primary colors into a colorimetric color reproduction image independent of the device are calculated. In calculating the conversion parameters, the exposure amount correction plate and the colorimetric conversion color chart are imaged in a frame sequential manner to obtain three primary color images, and the color image values of the exposure amount correction plate and the colorimetric conversion color chart are obtained. Each is normalized by the corresponding color image value of the exposure correction plate. As a result, the color images of the exposure correction plate and the colorimetric conversion color chart that are not affected by variations in shutter speed, fluctuations in the photographing light source, etc. during the surface sequential imaging of the exposure correction plate and the colorimetric conversion color chart are obtained. Trying to get. Based on the color image values of the exposure correction plate and colorimetric conversion color chart normalized in this way, and the colorimetric values measured for the exposure correction board and colorimetric conversion color chart, the standardization is performed. Conversion parameters for converting the color image values of the exposure correction plate and the color chart for colorimetric conversion into the colorimetric values are calculated.

  Subsequently, an arbitrary subject including the exposure amount correction plate is imaged in a frame sequential manner to obtain three primary color images, and each color image of the subject is normalized by the corresponding color image value of the exposure amount correction plate. As a result, a color image of the subject is obtained that is not affected by variations in shutter speed, fluctuations in the photographic light source, and the like during surface sequential imaging of the subject. The color image of the subject is converted into the colorimetric color reproduction image based on each color image of the subject standardized in this way and the calculated conversion parameter.

  The image processing method according to claim 1, wherein the subject having two or more misregistration detection marks is imaged in a frame sequential manner to obtain three primary color images, and each of the color images Including a step of detecting the position of the upper misalignment detection mark and a step of detecting an amount of misalignment between the color images based on the position of the detected misalignment detection mark. The color images of the three primary colors are characterized in that the positional deviation correction between the images is performed based on the detected positional deviation amount. Thereby, even if each color image of the three primary colors picked up in the surface sequential order is misaligned at the image forming position, the misalignment between the color images is corrected and no imaginary color or the like is generated.

  The image processing method according to claim 1 or 2, wherein the first normalization step includes an exposure amount correction plate and a colorimetry based on the three primary color images captured in the frame sequential manner. A step of displaying a screen including a conversion color chart on the display means, a step of designating each area of the exposure correction plate and the color measurement conversion color chart on the screen, and each color image for each of the designated areas The representative value for each color image in the area of the exposure amount correction plate and the colorimetric conversion color chart region is divided by the step of calculating the representative value of each and the representative value for each color image in the region of the exposure amount correction plate. And the divided value is used as the normalized exposure correction plate and color image value of the color chart for colorimetric conversion.

  4. The image processing method according to claim 1, wherein the second normalization step includes an object including an exposure amount correction plate based on the color image captured in the frame sequential manner. A step of displaying a screen on a display means, a step of designating an area of the exposure amount correction plate on the screen, a step of calculating a representative value for each color image of the designated region, and the exposure amount correction plate Dividing each color image captured in the frame sequential order by a representative value for each color image of the area of the plate.

  5. The image processing method according to claim 1, wherein the colorimetric color reproduction image is an XYZ color system XYZ image or a Lab color system Lab image. It is characterized by that.

  According to a sixth aspect of the present invention, there is provided an image processing system comprising a black and white imaging unit and a color filter of at least three primary colors, wherein the color filter unit sequentially switches and arranges each color filter on the photographing optical path of the black and white imaging unit, A frame sequential imaging apparatus having an imaging control unit for controlling the color filter unit and the monochrome imaging unit to acquire frame sequential image data during imaging, and an exposure amount correction plate imaged in the frame sequential order by the frame sequential imaging apparatus; And an image input means for inputting the three primary color images of the colorimetric conversion color chart and the subject's three primary color images including the exposure correction plate imaged in the frame order, the exposure correction plate and the measurement Each color image value of the color conversion color chart is normalized by the corresponding color image value of the exposure amount correction plate, and each color image of the subject is specified by the corresponding color image value of the exposure amount correction plate. And the standardization based on the standardized color image values of the exposure correction plate and the colorimetric conversion color chart and the colorimetric values measured on the exposure correction board and the colorimetric conversion color chart. Means for calculating a conversion parameter for converting the color image value of the exposure correction plate and the color chart for colorimetric conversion into the colorimetric value, the normalized color image of the subject, and the calculated conversion parameter And an image processing apparatus having means for converting the color image of the subject into the colorimetric color reproduction image.

  The invention according to claim 7 is an image processing program for obtaining a colorimetric color reproduction image from three primary color images imaged in a frame sequential manner, and an exposure amount correction plate and a colorimetric conversion color imaged in the frame sequential manner. A function of inputting color images of the three primary colors of the vote, a function of normalizing each color image value of the exposure amount correction plate and the color measurement conversion color sample with the corresponding color image value of the exposure amount correction plate, and the standard The exposure amount correction plate normalized based on the color image values of the exposure amount correction plate and the color chart for colorimetric conversion converted, and the colorimetric values measured for the exposure amount correction plate and the color chart for colorimetry conversion, and A function for calculating a conversion parameter for converting a color image value of a color chart for colorimetric conversion into the colorimetric value, and a color image of three primary colors of a subject including an exposure correction plate imaged in frame sequential order are input. Function and each color image of the subject corresponding to the exposure amount correction plate The computer realizes a function of normalizing with an image value and a function of converting the color image of the subject into the colorimetric color reproduction image based on the normalized color image of the subject and the calculated conversion parameter. It is characterized by letting.

  According to the present invention, a device-independent colorimetric color reproduction image such as an XYZ image of tristimulus values can be acquired, and in particular, a high-resolution colorimetric color reproduction image with high color reproducibility can be acquired. Can do.

  Hereinafter, preferred embodiments of an image processing method, an image processing system, and an image processing program according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing an embodiment of a frame sequential imaging apparatus constituting an image processing system according to the present invention.

  The frame sequential imaging device 10 includes an imaging device main body 100 and a rotating color filter device 200, and records a frame sequential color image captured through the rotating color filter device 200 on a memory card 154. Yes, the overall operation of the apparatus is centrally controlled by a central processing unit (CPU) 140. The ROM 151 stores programs, adjustment values, and the like in advance, and these programs, adjustment values, and the like are read as appropriate.

  The imaging apparatus main body 100 is provided with an operation unit 138 including a shutter button and a mode dial for setting a shooting mode, a reproduction mode, and the like. A signal corresponding to an operation on the operation unit 138 is input to the CPU 140. .

  Image light indicating a subject is imaged on a light receiving surface of a black and white image sensor 116 such as a CCD image sensor or a CMOS image sensor via a rotating color filter device 200, a photographing lens 112, and a diaphragm 114.

  The turret plate 202 of the rotating color filter device 200 is rotationally driven by a filter driving unit 137 and a motor 204 controlled by the CPU 140 so that each color filter is positioned on the optical axis of the photographing lens 112 for each frame sequential photographing. Be controlled.

  The photographing lens 112 is driven by a lens driving unit 136 controlled by the CPU 140, and focus control and the like are performed. The diaphragm 114 is composed of, for example, five diaphragm blades, and is driven by the diaphragm driving unit 134 controlled by the CPU 140. For example, the diaphragm 114 is controlled in five stages from the aperture values F2.8 to F11 in increments of 1 AV.

  In addition, the CPU 140 controls the diaphragm 114 via the diaphragm driving unit 134 and controls the charge accumulation time (shutter speed) in the monochrome image sensor 116 via the image sensor control unit 132.

  The signal charge accumulated in the monochrome image sensor 116 is read out as a voltage signal corresponding to the signal charge based on the readout signal applied from the image sensor control unit 132. The voltage signal read from the black and white image sensor 116 is applied to the analog signal processing unit 118, where the voltage signal for each pixel is sampled and held, amplified, and then applied to the A / D converter 120. The A / D converter 120 converts sequentially input analog signals into digital signals, and the digital signals (image data) are temporarily stored in a memory (SDRAM) 148 via the image input controller 122.

  The image data stored in the memory 148 is output to the VRAM 150 after required signal processing is performed via the digital signal processing unit 124. The VRAM 150 includes an A area and a B area each storing image data representing an image for one frame. In the VRAM 150, image data representing an image for one frame is rewritten alternately in the A area and the B area. The written image data is read from an area other than the area where the image data is rewritten out of the A area and B area of the VRAM 150. The image data read from the VRAM 150 is encoded by the video encoder 128 and output to the liquid crystal monitor 130 provided on the back of the camera, whereby the subject image is displayed on the display screen of the liquid crystal monitor 130.

  Further, when the shutter button of the operation unit 138 is first pressed (half-pressed), the AE operation and the AF operation are started. That is, the image data output from the A / D converter 120 is taken into the AF detection unit 142 and the AE detection unit 144. The AF detection unit 142 uses image data (for example, a signal in the center region (focus region) of the monochrome image sensor 116), and uses a high-pass filter (HPF) to extract high-frequency components from one-dimensional horizontal continuous image data. A value (AF evaluation value) obtained by extracting and integrating the high frequency components is output to the CPU 140. The AE detection unit 144 integrates image data of the entire screen, or integrates image data with different weights at the center and peripheral portions of the screen, and outputs the integrated value to the CPU 140.

  Based on the AF evaluation value input from the AF detection unit 142, the CPU 140 moves the taking lens 112 to the lens position where the AF evaluation value is maximized via the lens driving unit 136, and the integration input from the AE detection unit 144. The brightness of the subject (shooting Ev value) is calculated from the value, and the aperture value of the diaphragm 114 and the electronic shutter (shutter speed) of the black and white image sensor 116 are determined based on the shooting Ev value according to a predetermined program diagram. Based on the determined aperture value, the aperture 114 is controlled via the aperture drive unit 134, and the charge accumulation time in the monochrome image sensor 116 is controlled via the image sensor control unit 132 based on the determined shutter speed.

  When the AE operation and the AF operation are completed and the shutter button is pressed in the second stage (full press), frame-sequential imaging is performed in response to the depression, and the image data for each color in the frame sequence is the image data It is temporarily stored in the memory 148 via the input controller 122.

  The CPU 140 corrects the color shift of the image data for each color stored in the memory 148. Details of this color misregistration correction method will be described later.

  The digital signal processing unit 124 performs predetermined signal processing such as gain control processing, gamma correction processing, YC processing including offset processing, white balance correction, and sensitivity correction on the image data for each color after color misregistration correction. I do. The YC-processed image data (YC data) is read from the digital signal processing unit 124 and stored in the memory 148 again. Subsequently, the YC data is output to the compression / decompression processing unit 126, and a predetermined compression process such as JPEG (joint photographic experts group) is executed. The compressed YC data is again output to and stored in the memory 148 and then read out by the media controller 152 and recorded in the memory card 154.

  When RAW data recording is performed, the frame sequential RAW data input via the image input controller 122 is temporarily stored in the memory 148 and then directly stored as a RAW data file (the signal from the digital signal processing unit 124). It is recorded in the memory card 154 without being processed or compressed.

  FIG. 2 is a block diagram showing a hardware configuration example of an image processing apparatus constituting the image processing system according to the present invention.

  As shown in FIG. 2, the image processing apparatus 300 is composed of, for example, a personal computer or the like, and mainly stores a central processing unit (CPU) 310 that controls the operation of each component and a control program for the apparatus or executes a program. Data from a main memory 312 that is a working area, an operating system (OS) of a personal computer, an image processing program according to the present invention, various application software, a user image, and the like, and a CD-ROM A CD-ROM device 316 capable of reading data, a card interface 317 for reading image data (RAW data) from the memory card 154, a display memory 318 for temporarily storing display data, and image data from the display memory 318 , Display images and characters by character data A monitor device 320 such as an RT monitor or a liquid crystal monitor, a keyboard 322, a mouse 324 as a position input device, and signals such as the position of the mouse pointer on the monitor device 320 and the state of the mouse 324 by detecting the state of the mouse 324 Are transmitted to the CPU 310, a communication interface 327 for communicating with the frame sequential imaging device 10 and the like, and a bus 328 for connecting the above components.

  Note that the image processing apparatus 300 having the above-described configuration is well known except for an image processing program stored in the hard disk device 314, and therefore detailed description of each component will be omitted. The image processing program according to the present invention can be installed in the image processing apparatus 300 by setting a CD-ROM in which the image processing program is recorded in the CD-ROM device 316.

  In the image processing system according to the present invention, an XYZ color system XYZ image is acquired from an RGB image captured by the field sequential imaging device 10. The XYZ color system is one of the color systems that do not depend on the device defined by CIE (International Lighting Commission).

  FIGS. 3 to 10 are diagrams showing GUI (Graphical User Interface) screens showing operation procedures, operation contents, and the like displayed on the monitor device 320 of the image processing apparatus 300, respectively.

<Acquisition of frame sequential RGB image>
First, a whiteboard (see FIG. 5) on which misalignment detection marks (cross marks) are printed at the four corners is imaged in a frame sequential manner by the frame sequential imaging device 10, and RGB RAW data obtained by the imaging is stored in a memory. Record on card 154.

  Next, a white plate and 24 color charts (see FIG. 8) are imaged in the frame sequential order by the frame sequential imaging device 10, and RGB RAW data obtained by this imaging is recorded in the memory card 154. The white plate data is used to correct the exposure amount of each frame sequential image as will be described later. For this reason, it is not necessary to use a white plate, and a gray plate or, in the extreme, a color chart is also possible. In order to improve the correction accuracy, high reflectance and neutrality (spectral reflectance is flat) are required. A white plate is desirable in terms of satisfying these two conditions.

  Thereafter, the subject (actual scene) including the white plate is imaged in the frame sequential order by the frame sequential imaging device 10, and the RGB RAW data obtained by the imaging is recorded in the memory card 154. When imaging an actual scene, a white plate must be inserted for imaging.

  By loading the memory card 154 into the card interface 317, the image processing apparatus 300 can appropriately read out RGB RAW data. 2, the frame sequential imaging apparatus 10 is connected to a communication interface 327 such as IEEE1394 or USB, and the image processing apparatus 300 reads RAW data from the plane sequential imaging apparatus 10 via the communication interface 327. You may do it.

<Position evaluation operation procedure>
[Procedure 1]
In the image processing apparatus 300, the image processing program according to the present invention is activated.

[Procedure 2]
RGB RAW data obtained by imaging the cross mark is input. That is, as shown in FIG. 3, on the GUI screen, the R button is pressed, a file in which R RAW data is recorded is selected, and a file in which G and B RAW data is recorded is selected in the same manner.

[Procedure 3]
An “execute” button is pressed on the GUI screen of FIG. 3 to generate a honeycomb interpolated image. That is, the image processing apparatus 300 performs linear matrix processing, white balance processing, synchronization processing, and the like on the RAW data by RAW development software, and generates data that can be output to a display or the like. Note that the monochrome imaging element 116 of the frame sequential imaging device 10 is a pixel array called a honeycomb array, and every other center point of the geometric shape of the light receiving cell in the row direction and the column direction is half the pixel pitch ( (1/2 pitch). Therefore, in the RAW development, honeycomb interpolation for converting the pixel data at the spatial position of the honeycomb array into data having no pitch deviation is also performed.

[Procedure 4]
Next, on the GUI screen shown in FIG. 4, a “positional deviation evaluation” button is pressed to launch a deviation correction window.

[Procedure 5]
FIG. 5 is a GUI screen showing a deviation correction window. In the four windows of the GUI screen, images of the four corners of the whiteboard having cross marks are displayed. Then, a position (cross mark) to be evaluated for positional deviation is selected from the four windows by the following procedure, and the amount of positional deviation is calculated.

  (1) Move the rectangle to the cross mark.

  (2) Press the “OK” button on the displayed window.

  (3) Perform the above operations (1) and (2) at four locations.

  (4) After confirming, press the “Calculate” button.

  (5) Deviation amounts based on the G channel are calculated for the R-AFFINE coefficient (R channel) and the B-AFFINE coefficient (B channel). That is, the positional deviation and the angular deviation of the R image and the B image are calculated on the basis of the G image. Thereby, the mutual position shift of RGB image can be evaluated. When a positional deviation of one pixel or more occurs, the deviation of the R image and the B image with respect to the G image can be corrected based on the calculated deviation amount.

<Operation procedure of calibration system and display system>
[Procedure 1]
On the GUI screen of FIG. 6, RGB RAW data obtained by imaging a white plate and a color chart is input. That is, on the GUI screen, the “R” button is pressed, a calibration file in which R RAW data is recorded is selected, and a file in which G and B RAW data are recorded is selected in the same manner.

[Procedure 2]
Press the “XYZ” button and input the XYZ file of the white plate and color chart. In this XYZ file, 25 XYZ values measured by a spectral luminance meter of a white plate and 24 color charts are recorded.

[Procedure 3]
When the “execute” button is pressed, the input file is read.

[Procedure 4]
Next, on the GUI screen shown in FIG. 7, the “MTX calculation” radio button is checked, and the “OK” button is pressed to launch a range designation window.

[Procedure 5]
FIG. 8 is a GUI screen showing an average value calculation window for the selected range. On this GUI screen, the “individual designation” radio button is checked, and the X size and Y size of the rectangle from which data is read are input. Thereafter, the positions of the white plate and the 24 color charts are sequentially selected, and the “RGB value acquisition / MTX calculation” button is pressed. If the average value is displayed in the rectangle, the “OK” button is pressed to close the window. The order of selecting the white plate and the color chart is matched so that the order of the XYZ values of the white board and the color chart is the same as the order of obtaining the RGB values of the white board and the color chart. In this example, a white plate is first selected, and then 24 color charts are selected from the lower left to the upper right of the screen on the GUI screen of FIG.

  By the above operation, the average value of the rectangular area of the white plate and the color chart of 24 colors is calculated for each surface of RGB, and the RGB image is converted into an XYZ image based on this calculation result and the XYZ values of the white board and the color chart. A conversion parameter (conversion matrix) to be converted into is calculated. Details of this will be described later. In this embodiment, the average value of the rectangular areas of the white plate and the color chart is the representative value of the color image of the rectangular area. However, the present invention is not limited to this, and a value such as a median value may be used.

[Procedure 6]
Next, RGB RAW data obtained by imaging a subject (real scene) including a white plate is input on the GUI screen of FIG. That is, on the GUI screen, the “R” button is pressed to select a file for an actual scene in which R RAW data is recorded. Similarly, a file in which G and B RAW data is recorded is selected.

  As the actual scene file, a file in which RAW data imaged under the same light source as the calibration file is recorded is used. When using RAW data imaged under different light sources, it is necessary to calculate the conversion matrix again.

[Procedure 7]
Next, a “white plate and light source XYZ input” button is pressed on the GUI screen of FIG. 9 to launch the white plate RGB acquisition viewer shown in FIG. On the GUI screen shown in FIG. 10, “White plate designation” is checked and an appropriate rectangular size is input. Subsequently, the position of the white plate on the GUI screen is designated with the mouse.

[Procedure 8]
Subsequently, when the “OK” button is pressed, an average value is calculated for each RGB image in the input rectangle, and the RAW data of the actual scene is normalized based on the calculation result. As a result, correction of shutter variation (= corresponding to white balance correction) at the time of imaging of each surface of RGB is performed.

  Then, the RGB image of the actual scene is converted into an XYZ image based on the RGB RAW data of the actual scene, the RGB value of the white plate, the XYZ value, and the conversion matrix.

  FIG. 11 is a flowchart showing a flow of processing for acquiring an XYZ image from an RGB image captured by the field sequential imaging device 10 by two processes (calibration system A and display system B) in the image processing system according to the present invention.

  In FIG. 11, as described above, the RAW data (calibration RAW data) of the white plate and the 24 color charts captured in the frame sequence by the frame sequential imaging apparatus 10 and the RAW data (actual data) of the subject including the white plate. (RAW data of scene) is captured (step S10). These RAW data are subjected to honeycomb interpolation and temporarily stored as new calibration RAW data and real scene RAW data (steps S12 and S14).

  The RAW data is used for RAW image display, and RGB values of the white plate and 24 color charts are calculated from the RAW data for calibration. The RGB values of the white plate are calculated from the RAW data of the actual scene. Is calculated (step S16). The RGB values of the white plate and color chart calculated from the RAW data for calibration are sent to the calibration system A, and the RGB values of the white plate calculated from the RAW data of the actual scene and the RAW data of the actual scene are respectively displayed. Sent to system B.

<Calibration system A>
In the calibration system A, a parameter (MTX) file for converting an RGB image into an XYZ image is created by the following processing.

  (1) XYZ (Xn, Yn, Zn), RGB (Rn, Gn, Bn) of 24 color charts and XYZ (Xw, Yw, Zw), RGB (Rw, Gw, Bw) of white plate Input (steps S20 and S22). The XYZ (Xn, Yn, Zn) of the 24 color charts and the XYZ (Xw, Yw, Zw) of the white plate are obtained by calculating from spectral data and color matching functions obtained in advance by a spectral luminance meter. Colorimetric values.

  (2) The RGB value and XYZ value input in (1) above are substituted into the following equation to calculate a conversion matrix A (3 × 3) (step S24).

As is clear from the above equation (1), the input RGB values are not affected by variations in shutters during frame-sequential shooting, fluctuations in the shooting light source, shutter speed, and aperture change. Normalization is performed by dividing by values (Rw, Gw, Bw).

  The conversion matrix A thus obtained is output to the display system B as a parameter file (step S26).

<Display system B>
In the display system B, an XYZ image is created from the RGB image of the actual scene.

  (1) The RGB image (RAW data) of the actual scene, the RGB value of the white plate, and the parameter file created by the calibration system A are input (step S30).

  (2) An XYZ image is calculated by substituting the input value into the following equation (2) (step S32).

As is clear from the above equation (2), the RGB values of the actual scene are set to the values of the white plate so that they are not affected by variations in shutters during field sequential shooting, fluctuations in the shooting light source, shutter speed, and aperture change. Normalization is performed by dividing by RGB values (Rw ′, Gw ′, Bw ′). In addition, in order to correct fluctuations in shooting conditions between shooting for calibration and shooting of the actual scene, the XYZ values (Xw ′, Yw ′, Zw ′) of the white plate of the actual scene are used as the white color used during calibration. Divide by the XYZ values (Xw, Yw, Zw) of the plate. Note that the XYZ values (Xw ′, Yw ′, Zw ′) of the white plate of the actual scene are the RGB values (Rw ′, Gw ′, Bw ′) of the conversion matrix A calculated by the equation (1) and the white plate of the actual scene. ).

  Further, when the XYZ values of the white plate in the calibration system A and the display system B do not change (when the position of the white plate and the light source do not change), (Xw ′, Yw ′, Zw ′) = (Xw, Yw, Zw), the equation (2) becomes the following equation (3).

Further, the XYZ values (Xs ′, Ys ′, Zs ′) of the photographing light source are calculated from the XYZ values (Xw ′, Yw ′, Zw ′) of the white plate of the actual scene (step S34). In equations (2) and (3), by multiplying by 100 / Ys ′, an XYZ image standardized with Ys ′ of the imaging light source as 100 can be obtained.

  When the color difference (ΔE) was calculated from the XYZ image acquired in this way, it was about ΔE = 1.5. This confirmed that a high-resolution and high-precision colorimetric color reproduction image (XYZ image) can be obtained. The XYZ image obtained by this system can be expected to be used in fields such as product color evaluation and printing.

  In this embodiment, the case where an XYZ image is obtained from a frame sequential RGB image has been described. However, the present invention can also be applied to a case where a colorimetric color reproduction image (for example, a Lab image) other than an XYZ image is obtained. . The frame sequential color image is not limited to an RGB image, and may be a frame sequential CMY image using a CMY color filter, for example. Furthermore, the present invention can also be applied to a case where a colorimetric color reproduction image is acquired from a frame sequential color image using four or more color filters. In the case of N (an integer of N ≧ 4), an N × 3 conversion matrix is used as the colorimetric correction conversion.

FIG. 1 is a block diagram showing an embodiment of a frame sequential imaging apparatus constituting an image processing system according to the present invention. FIG. 2 is a block diagram showing a hardware configuration example of an image processing apparatus constituting the image processing system according to the present invention. FIG. 3 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus. FIG. 4 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus. FIG. 5 is a view showing a GUI screen displayed on the monitor device of the image processing apparatus, and more particularly a view showing a GUI screen showing a shift correction window. FIG. 6 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus. FIG. 7 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus. FIG. 8 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus, and more particularly a diagram showing a GUI screen showing an average value calculation window for a selection range. FIG. 9 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus. FIG. 10 is a diagram showing a GUI screen displayed on the monitor device of the image processing apparatus, and more particularly a diagram showing a white board RGB acquisition viewer. FIG. 11 is a flowchart showing a flow of processing for acquiring an XYZ image from a captured RGB image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Frame sequential imaging device, 100 ... Imaging device main body, 200 ... Rotation color filter device, 300 ... Image processing apparatus

Claims (7)

In an image processing method for obtaining a colorimetric color reproduction image from color images of three primary colors imaged in frame sequential order,
Capturing an exposure amount correction plate and a colorimetric conversion color chart in a surface sequence to obtain a color image of three primary colors;
A first normalization step of normalizing each color image value of the exposure amount correction plate and the colorimetric conversion color chart with a corresponding color image value of the exposure amount correction plate;
The normalized exposure amount correction based on the color image values of the normalized exposure amount correction plate and the colorimetric conversion color chart and the colorimetric values measured for the exposure amount correction plate and the colorimetric conversion color chart. Calculating a conversion parameter for converting the color image values of the plate and the color chart for colorimetric conversion into the colorimetric values;
Capturing a subject including an exposure correction plate in a surface sequential manner to obtain a color image of three primary colors;
A second normalization step of normalizing each color image of the subject with a corresponding color image value of a colorimetric conversion color chart;
Converting the color image of the subject into the colorimetric color reproduction image based on the normalized color image of the subject and the calculated conversion parameter;
An image processing method comprising:   A step of capturing a subject having two or more misregistration detection marks in a surface sequential manner to obtain a color image of three primary colors, a step of detecting a position of the misregistration detection mark on each color image, and the detection Detecting a displacement amount between the color images based on the position of the displacement detection mark, and thereafter, the three primary color image images that are imaged in the frame order are based on the detected displacement amount. The image processing method according to claim 1, wherein a positional deviation correction between the images is performed.   The first normalization step includes a step of causing a display unit to display a screen including an exposure amount correction plate and a color chart for colorimetric conversion based on the color images of the three primary colors captured in the frame order, and on the screen. A step of designating each region of the exposure amount correction plate and the color chart for colorimetric conversion, a step of calculating a representative value for each color image for each designated region, and each color image of the region of the exposure amount correction plate Dividing the exposure value correction plate and the representative value for each color image in the colorimetric conversion color chart area by the representative value for each, and the normalized exposure amount correction plate and the measurement value. 3. The image processing method according to claim 1, wherein the color image value of the color chart for color conversion is used.   The second normalization step includes a step of displaying a subject screen including an exposure amount correction plate on a display unit based on the color image picked up in the frame sequential order, and designates an area of the exposure amount correction plate on the screen. A step of calculating a representative value for each color image of the designated region, and a step of dividing each color image captured in the surface sequential order by a representative value for each color image of the region of the exposure correction plate The image processing method according to claim 1, further comprising:   5. The image processing method according to claim 1, wherein the colorimetric color reproduction image is an XYZ color system XYZ image or a Lab color system Lab image. 6. A black-and-white imaging means, a color filter means having color filters of at least three primary colors, each color filter being sequentially switched and arranged on the photographing optical path of the black-and-white imaging means, and the color filter means and the monochrome imaging means when imaging a subject A field-sequential imaging device having an imaging control means for controlling the image acquisition to acquire field-sequential image data;
Three primary color images of an exposure amount correction plate and colorimetric conversion color chart imaged in the frame sequential order by the frame sequential imaging device and an exposure amount correction plate imaged in the frame sequential manner. Standardizing the color image values of the exposure amount correction plate and the colorimetric conversion color chart with the corresponding color image values of the exposure amount correction plate, respectively, and each color image of the subject. Means for normalizing with the corresponding color image value of the exposure amount correction plate, color image values of the normalized exposure amount correction plate and colorimetric conversion color chart, and the exposure amount correction plate and colorimetric conversion color chart Means for calculating a conversion parameter for converting the standardized exposure correction plate and color image values of the colorimetric conversion color chart into the colorimetric values based on the measured colorimetric values; The subject color image and the calculated conversion parameter And an image processing apparatus having a means for converting the color image of the object on the colorimetric color reproduction image Zui,
An image processing system comprising: In an image processing program for obtaining a colorimetric color reproduction image from color images of three primary colors imaged in frame sequential order,
A function for inputting color images of the three primary colors of an exposure amount correction plate and a color chart for colorimetric conversion, which are imaged in frame sequential order;
A function of normalizing each color image value of the exposure amount correction plate and the color chart for colorimetric conversion with a corresponding color image value of the exposure amount correction plate;
The normalized exposure amount correction based on the color image values of the normalized exposure amount correction plate and the colorimetric conversion color chart and the colorimetric values measured for the exposure amount correction plate and the colorimetric conversion color chart. A function of calculating a conversion parameter for converting the color image values of the board and the color chart for colorimetric conversion into the colorimetric values;
A function of inputting a color image of three primary colors of a subject including an exposure correction plate imaged in a frame sequential manner;
A function of normalizing each color image of the subject with a corresponding color image value of the exposure correction plate;
A function of converting the subject color image into the colorimetric color reproduction image based on the standardized subject color image and the calculated conversion parameter;
An image processing program for causing a computer to realize the above.
114