JP2011259047A - Color correction device, color correction method, and video camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a color correction device and a color correction method capable of computing a suitable color correction parameter even when a gamut error or an observation error exists, and applying that parameter to a picked-up image by a video camera to correct colors.SOLUTION: The color correction device comprises: a reference storage portion storing a color reference signal; a buffer inputting a re-photograph monitor photographing signal obtained by photographing a re-photograph monitor which displays a color reference image corresponding to the color reference signal; a comparing/computing portion comparing the re-photograph monitor photographing signal stored in the buffer with the color reference signal stored in the reference storage portion, and calculating a correction parameter so as to reduce a difference between the re-photograph monitor photographing signal and the color reference signal; a correction parameter storage portion storing the calculated correction parameter; and a correction processing portion carrying out correction processing to an inputted image data on the basis of the correction parameter stored in the correction parameter storage portion.

Description

  The present invention relates to a color correction device, a color correction method, and a video camera system.

  When shooting using a plurality of broadcast station video cameras, strict color matching is required between the video cameras to be used. In general, it is known that such color matching between video cameras is manually adjusted in advance by staff such as a broadcasting station.

  In a video camera for a broadcasting station, the spectral characteristics of a color filter and a color separation prism arranged in front of an image sensor such as a CCD (Charge Coupled Device) are slightly different for each video camera. For this reason, when the hue between the video cameras is not corrected, even when shooting the same object, the hue of the output video that has been shot differs between the video cameras.

  As a technique for performing color matching between a plurality of video cameras, a reference imaging device and another imaging device that matches the hue with respect to the reference imaging device have a predetermined same color, for example, in a color chart. A correction method has been proposed in which each color is photographed under the same conditions, and photographing data obtained as a result of photographing is compared, and color data output from another video camera whose hue is to be matched is appropriately corrected.

  Another object of the present invention is to realize a color correction device that can easily perform color matching between a plurality of video cameras by an operation for each video camera without requiring a reference video camera, Reference color information generation means for generating reference color information for a predetermined color is provided, and based on information generated by the reference color information generation means and a signal obtained by imaging a predetermined color with a video camera A proposal for adjusting a color based on a signal obtained by imaging with a video camera to a reference color by providing a correction unit that corrects a signal obtained by imaging with a video camera according to the determined correction content is, for example, It is disclosed in Patent Document 1.

Japanese Patent Laid-Open No. 10-285610 K. Kanatani, Statistical Optimization for GeometricComputation: Theory and Practice, Elsevier Science, Amsterdam, The Netherlands (1996); reprinted DoverPublications, New York, U.S.A. (2005). Kenichi Kanaya, Yasuyuki Shibuya, Extended FNS Method for Constrained Parameter Estimation, IPSJ Research Report, 2007-CVIM-158-4, (2007-3), Kagoshima City (Kagoshima Univ.), 25-32. Hirotaka Niizuma and Kenichi Kanaya, New computational algorithm for optimal projective transformation, Information Processing Society of Japan, 2009-CVIM-169-37, (2009-11), Kanazawa City (Kanazawa Institute of Technology), 1-8.

  However, when the operator performs color correction manually, considerable skill and man-hours are required. In addition, when the color correction parameter is calculated using a simple least square method, there is a concern that an accurate correction parameter cannot be calculated.

  In particular, calculation of a correction parameter that is considered so as to correct a gamut error and an observation error has not been conventionally performed, and there has been a problem that white balance and black balance cannot be achieved.

  The present invention has been made in view of such problems. Even when gamut errors and observation errors exist, an appropriate color correction parameter is calculated and applied to an image captured by a video camera. It is an object of the present invention to provide a color correction apparatus and a color correction method that can correct this.

  The color correction apparatus of the present invention includes a reference storage unit that stores a color reference signal, a buffer that receives a reshooting monitor shooting signal obtained by shooting a reshooting monitor that displays a color reference image corresponding to the color reference signal, Comparison operation for comparing the re-shooting monitor shooting signal stored in the buffer with the color reference signal stored in the reference storage unit and calculating a correction parameter so as to reduce the difference between the re-shooting monitor shooting signal and the color reference signal A correction parameter storage unit that stores the calculated correction parameter, and a correction processing unit that performs correction processing based on the correction parameter stored in the correction parameter storage unit for the input video data. It is characterized by.

  The color correction apparatus of the present invention preferably displays the color reference image corresponding to the color reference signal on the reshooting monitor by reading the color reference signal from the reference storage unit and outputting it to the reshooting monitor. And

  In the color correction apparatus of the present invention, it is more preferable that the correction processing unit reads the color reference signal stored in the reference storage unit, performs correction processing based on the correction parameter on the color reference signal, and performs a verification signal. Is output to the re-shoot monitor, the re-shoot monitor shooting verification signal obtained by shooting the re-shoot monitor that displays the verification signal is input to the buffer, and the comparison calculation unit is configured to receive the color reference signal stored in the reference storage unit and the buffer. Is compared with the re-shooting monitor photographing verification signal stored in the above, and the presence or absence of the difference is verified.

  Further, the video camera system of the present invention provides any of the color correction devices described above, a re-shooting monitor that displays a color reference image corresponding to the color reference signal, and a re-shooting monitor shooting signal obtained by shooting the re-shooting monitor. And a video camera for outputting to the correction device.

  The color correction method of the present invention is a color correction method for correcting the hue of an image captured by a reshooting monitor using any one of the color correction devices described above, and corresponds to a color reference signal for the reshooting monitor. A step of displaying a color reference image, a step of acquiring a reshooting monitor shooting signal from a video camera that has shot the color reference image displayed on the reshooting monitor, the acquired reshooting monitor shooting signal, and storing in the reference storage unit And a step of calculating a correction parameter by comparing the color reference signal and a step of performing a correction process on a video signal input using the calculated correction parameter.

  In the color correction method of the present invention, preferably, in the step of displaying the color reference image corresponding to the color reference signal on the re-shooting monitor, the color correction device outputs the color reference signal stored in the reference storage unit and outputs it. It is a step of displaying a color reference image based on a re-shooting monitor.

  Even when gamut errors or observation errors exist, it is possible to provide a color correction device and a color correction method that can calculate appropriate color correction parameters and apply them to captured images by a video camera to correct them. .

It is a conceptual diagram explaining the video camera system in the case of calculating the correction parameter which should be applied with respect to the video signal output to a re-shooting monitor. It is a conceptual diagram explaining the video camera system in the case of verifying whether the correction parameters calculated and stored by the color correction apparatus are appropriate. It is a conceptual diagram explaining the video camera system in the case of performing an imaging process of an actual re-shooting monitor using a color correction apparatus for which calculation and verification of correction parameters have been completed in advance. It is a conceptual diagram explaining the comparison calculating part of a color correction apparatus further in detail. It is a schematic diagram explaining the processing flow of the color correction apparatus in a video camera system. It is a figure explaining the RMS error of the calculated three-dimensional affine transformation. (A) is an ideal ramp (ramp) image and waveform display, (b) is a color conversion image and waveform display, (c) is a color correction model based on a linear expression when clipped data is excluded, and (d) is Color correction image and waveform display by color correction model (c), (e) Polynomial color correction model with level constraint, (f) Color correction image and waveform display by quaternary polynomial model of color correction model (e) It is a figure explaining. It is an outline figure explaining a case where standard known color charts, such as a Macbeth chart, are photoed with a plurality of video cameras, and are corrected using a color correction device. It is an example (three-dimensional affine transformation) of a color conversion model. It is an example of a color conversion model (three-dimensional affine transformation + clip processing).

  The color correction apparatus according to the present embodiment automatically recognizes the color level from an image obtained by photographing a color chart by image processing in order to automate the color adjustment of a plurality of re-shooting monitors and cameras.

  Color correction parameters for correcting the output to the reshoot monitor so that the acquired color level is the original color level of a known color chart, taking into account the effects of color level observation errors and gamut errors in the color space Thus, a color correction device that performs optimal calculation and correction processing is obtained.

  In addition, the color correction apparatus according to the embodiment does not perform color correction depending on the appearance observation by hand unlike the prior art, and does not calculate color correction parameters by the least square method. For this reason, the color correction apparatus according to the embodiment cannot accurately calculate the color correction parameter due to the influence of the observation error and the gamut error, and particularly has a conventional problem that the white balance and the black balance cannot be obtained due to the gamut error. can be solved.

  Even when there is a gamut error, an optimum model is selected and applied from the result of calculation assuming a plurality of color correction models (multi-constraint FNS method or multi-constraint extended FNS method), and preferably white level or black level is selected. It is possible to realize a color correction apparatus that performs arithmetic processing with level restrictions.

With this color correction device, it is possible to correct the upper limit or lower limit video data cut by clipping processing such as so-called overexposure to natural colors, that is, it can also handle gamut errors. It becomes.
(Outline of the embodiment)

  FIG. 1 is a conceptual diagram illustrating a video camera system 1000 in the case of calculating correction parameters to be applied to a video signal output to a re-shooting monitor.

  In FIG. 1, a re-shooting monitor 200 is installed in a studio or the like, for example, and is rendered by a plurality of lights 250 (1) and 250 (2). In addition, the color correction apparatus 100 includes a reference storage unit 150 that stores a color reference signal in advance.

  The color reference signal may be, for example, an RGB signal intensity that represents a known reference hue, and typically each of a plurality of colors obtained by photographing a calibration target such as a Macbeth color chart under a predetermined condition. Reference signal.

  The re-imaging monitor 200 displays the color reference signal acquired from the color correction apparatus 100 as a color reference image 210. In addition, the video camera 300 outputs a recapture monitor shooting signal obtained by shooting the recapture monitor 200 on which the color reference image 210 is displayed to the color correction apparatus 100.

  The color correction apparatus 100 stores the recapture monitor shooting signal acquired from the video camera 300 in the frame buffer 140. Further, the color correction apparatus 100 compares the color reference signal stored in the reference storage unit 150 with the recapture monitor image capture signal stored in the frame buffer 140, and calculates a difference between the recapture monitor image capture signal and the color reference signal. In order to reduce, a comparison operation unit 130 is provided that calculates a correction parameter to be performed on the video signal to be applied to the re-shooting monitor 200.

  The color correction apparatus 100 also includes a correction parameter storage unit 110 that stores the correction parameters calculated by the comparison calculation unit 130. In addition, the color correction apparatus 100 includes a correction processing unit 120 that performs correction processing based on the correction parameters stored in the correction parameter storage unit 110 and outputs the input captured video or VTR video.

  As a result, the VTR video or the like input to the color correction apparatus 100 is appropriately corrected for the gamut error and the observation error based on the correction parameter stored in the correction parameter storage unit 110 in the correction processing unit 120, and the re-shooting monitor. It is output to 200 and displayed. Therefore, the display screen of the re-shooting monitor 200 included in the video shot by the video camera 300 can be viewed as a proper video with natural colors.

  FIG. 2 is a conceptual diagram illustrating the video camera system 2000 when verifying whether or not the correction parameters calculated and stored by the color correction apparatus 100 are appropriate. In FIG. 2, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted here in order to avoid duplication of explanation.

  As illustrated in FIG. 2, the correction processing unit 120 of the color correction apparatus 100 uses the correction parameters stored in the correction parameter storage unit 110 to perform correction processing on the color reference signal stored in advance in the reference storage unit 150. Carry out.

  The corrected color reference signal is output as a verification signal to the re-imaging monitor 200 and is displayed on the re-imaging monitor 220 as a verification image 220 corresponding to the output verification signal. The verification image 220 displayed on the recapture monitor 220 is captured by the video camera 300 and fed back to the color correction apparatus 100 as a recapture monitor capture verification signal.

  The comparison operation unit 130 compares the color reference signal stored in the reference storage unit 150 with the re-shooting monitor shooting verification signal fed back from the video camera 300, and the re-shooting monitor shooting verification signal detects a color shift from the color reference signal. Verify whether it has occurred. The comparison calculation unit 130 may output the verification result to notify an operator such as a video engineer.

  By this processing, it is possible to verify whether or not the correction parameter is appropriate. The correction parameters are expected to vary depending on the arrangement environment of the re-imaging monitor 200, for example, the type, number, and position of the lights 250 (1) and 250 (2) and the characteristics of the re-imaging monitor 200 itself. For this reason, it is preferable that the color correction apparatus 100 appropriately performs verification processing to check whether the correction parameter is appropriate in each scene.

  Further, the re-shooting monitor shooting verification signal shot and output by the video camera 300 is input to the master monitor 400 and displayed, and the re-shooting monitor 200 displaying the verification video 220 is displayed on the display screen. It is displayed as a verification video 410.

  The recapture monitor verification video 410 is a video reflecting the color of the video that the viewer should see. That is, the color correction apparatus 100 corrects the screen image of the re-shooting monitor 200 included in the output image from the video camera 300 viewed by the viewer so as to be displayed as a natural hue.

  As a result, for example, in a news program, when another re-shooting monitor installed in the background of the newscaster is included in the screen viewed by the viewer, it is displayed in the re-taking monitor, that is, in a small screen within the screen. It is possible to realize a color correction apparatus that performs a correction process so that a color tone is natural for an image.

  FIG. 3 is a conceptual diagram for explaining a video camera system in a case where an actual re-shooting monitor shooting process is performed using the color correction apparatus 100 for which correction parameters have been calculated and verified in advance.

  The color correction apparatus 100 receives a VTR video or a video shot by another video camera. The input video is subjected to color correction processing in the correction processing unit 120 using the correction parameters stored in the correction parameter storage unit 110.

  The input video that has undergone the color correction processing is transmitted from the color correction apparatus 100 to the re-shooting monitor 200 and displayed on the screen 230. The re-imaging monitor 200 is installed in the background of a news caster or the like, for example, and is rendered by the lights 250 (1), 250 (2) and the like.

  The re-shooting monitor 200 on which the effect has been given is shot by the video camera 300 together with the news caster. Usually, the production effect is adjusted by targeting a real tangible object in the studio (for example, a caster's clothes or a person itself). Therefore, the screen display 230 in the re-shooting monitor 200 photographed under the same production is generally used. It is known that when shooting with the video camera 300, the hue becomes unnatural.

  However, the color correction apparatus 100 uses the input video corrected using the correction parameters calculated in advance so that the screen display 230 of the re-shooting monitor 200 included in the video captured by the video camera 300 has a natural hue. Is displayed on the re-imaging monitor 200.

  Therefore, as shown in FIG. 3, when a video captured by the video camera 300 is displayed on the master monitor 400, a recapture monitor input video 420 displayed on the recapture monitor 200 included in the screen of the master monitor 400. Therefore, there is no need for a manual correction process by a video engineer or the like, and a natural color screen is displayed.

  1 to 3, for convenience of explanation, the case where there is one video camera 300 has been described. However, a plurality of video cameras 300 may be provided. In the case of a plurality of cameras, it is preferable that the color correction between the video cameras 300 is separately adjusted in advance so that there is no difference between them.

  1 to 3, the case where there is one re-imaging monitor 200 has been described for convenience of explanation, but a plurality of re-imaging monitors 200 may be provided. An appropriate correction parameter corresponding to each of the plurality of re-imaging monitors 200 is calculated by the color correction apparatus 100 and stored. Then, the color correction device 100 may output and display video signals that have been corrected independently from each of the re-shooting monitors 200. In this case, a plurality of correction parameter storage units 110 may be provided corresponding to each recapture monitor 200, and a plurality of other functions of the color correction apparatus 100 may be provided corresponding to each recapture monitor 200. Also good.

  In addition, when the characteristics, arrangement environment, effects, etc. of the plurality of re-shooting monitors 200 are exactly the same, and processing with the same correction parameters is preferable, the same correction parameters are applied to the plurality of re-shooting monitors 200. Then, the corrected video may be output and displayed.

  FIG. 4 is a conceptual diagram for explaining the comparison operation unit 130 of the color correction apparatus 100 in more detail. As shown in FIG. 4, the comparison calculation unit 130 includes a multiple model calculation unit 131 based on the multi-constraint FNS method or the multi-constraint extended FNS method.

  The multiple model calculation unit 131 based on the multi-constraint FNS method or the multi-constraint extended FNS method is assumed to calculate a correction model using at least one of the multi-constraint FNS method or the multi-constraint extended FNS method. The calculated correction model may be, for example, a three-dimensional translation conversion model, a rigid body conversion model, a similarity conversion model, an affine conversion model, a projective conversion model, or the like.

  The multi-constrained FNS method can correct the observation error satisfactorily, and the multi-constrained extended FNS method uses the black level constraint or the white level constraint, so that the gamut error can be corrected well in addition to the observation error. The multiple model calculation unit 131 based on the multi-constrained FNS method or the multi-constrained extended FNS method calculates an optimal solution by repeating the least square method as an initial value. For this reason, as the numerical calculation, the calculation of the eigenvalue / eigenvector of the matrix is repeatedly executed until the convergence determination condition is satisfied. On the other hand, in the case of the conventional method of least squares, the calculation is completed with only one eigenvalue calculation.

  In addition, the comparison operation unit 130 includes a selection criterion calculation unit 133 that determines a criterion for selecting an optimum model from among the plurality of calculated models. The selection criterion calculation unit 133 calculates a criterion such that a model with a small number of parameters and parameters is selected so that the model selection is not performed only by the goodness of the models (that is, the large number of parameters). To do.

  More specifically, the selection criterion calculation unit 133 calculates any one of geometric AIC and geometric MDL as a selection criterion.

  The model selection is described in detail in Kanaya's paper here, and some basic concepts will be briefly explained below.

  "I want to fit a straight line, a quadratic curve, a cubic curve, or a higher order curve to a plurality of points given on the plane. What should I apply? The simple idea is First, a straight line, a quadratic curve, a cubic curve,... Are applied in order, and the one having the smallest discrepancy with the data point (what is quantitatively evaluated is called “residual”) is selected. But this doesn't work. This is because higher-order curves are better applied to data points, and if a sufficiently high-order curve is selected, one that passes through all data points (residual is zero) can be obtained. This does not meet the original purpose. This is because the purpose of fitting the curve is to estimate the "true curve" behind it from a small number of data with errors. In general, a mathematical formula having an unknown parameter that is introduced to estimate a true structure from erroneous data is called a “model”. Naturally, the larger the number of parameters to be adjusted (this is called “degree of freedom”), the better the data is matched (ie, the residual is reduced). And the greater the degree of freedom, the better the model fits the data error. How can we prevent this and choose the most appropriate model? This is the issue of “model selection”. (Regarding model selection criteria) The above problem can also be considered as follows. In curve fitting, the straight line is a special case where the quadratic coefficient of the quadratic curve is zero. Similarly, a quadratic curve is a special case of a cubic curve, and a cubic curve is a special case of a quartic curve. That is, each set of curves of degree has an inclusive relationship, a set of straight lines is a subset of a set of quadratic curves, a set of quadratic curves is a subset of a set of cubic curves, and any order The set of curves is also a subset of the higher order set. On the other hand, the residual corresponds to the “distance” between the observed data and the model. The model that minimizes the residual is the model closest to the observed data. However, if the model has an inclusive relationship, the closest model is of course chosen from the largest set. This is because the distance does not increase or decrease when limited to a subset. For this reason, in order to select an element of the subset, not only the distance but also some evaluation that gives priority to a model having a smaller degree of freedom is necessary. This is measured by the “model selection criteria” and generally takes the following form. (Residual) + (penalty for degree of freedom) If a model with a high degree of freedom is selected in order to reduce the residual of the first term, the second term becomes larger. On the other hand, if a model that minimizes the sum of the two is selected, a model that balances the two is selected. Various model selection criteria such as “Akaike AIC”, “Schwarz BIC”, “Rissanen MDL”, “Mallows Cp” have been proposed. However, which model is selected depends on the standard to be used and there is no absolute one ”(Kenya Kanaya, Understanding of moving images by model selection, TELECOM FRONTIER, No. 44, August 2004, pp. 4-10.) .

  In addition, the comparison operation unit 130 includes a color correction model selection unit 132 that selects a color correction model to be applied in accordance with the criterion calculated by the selection criterion calculation unit 133. The color correction model selection unit 132 performs calculation according to the criterion calculated by the selection criterion calculation unit 133, and selects a model in which the calculated result is the minimum value. Intuitively, it can be said that the model with the smallest calculation result is a balanced model with a relatively small number of parameters and good fitting.

  In addition, the comparison calculation unit 130 includes a correction parameter calculation unit 134 that calculates a correction parameter based on the selected color correction model. For example, in the RGB color space, the correction parameters calculated by the correction parameter calculator 134 may be a total of nine correction parameters for each of the white level, black level, and gamma level of RGB.

  FIG. 5 is a schematic diagram illustrating a processing flow of the color correction apparatus 100 in the video camera system 1000. Hereinafter, the processing of the color correction apparatus 100 will be described sequentially according to the flow shown in FIG.

(Step S510)
The color correction apparatus 100 outputs the color reference signal stored in the reference storage unit 150 to the re-imaging monitor 200. As the color reference signal, an image signal of a calibration target composed of a plurality of color regions used for color correction in the video industry, such as a Macbeth chart image signal or a color bar image signal, can be used.

  In addition, another reference output device that stores the same color reference signal as the color reference signal stored in the reference storage unit 150 is provided separately, and the re-shooting monitor 200 is replaced by the reference output device instead of the output from the color correction device 100. The color reference signal may be supplied to.

(Step S520)
The re-imaging monitor 200 displays a color reference image 210 corresponding to the color reference signal input from the color correction apparatus 100.

(Step S530)
The video camera 300 shoots the re-imaging monitor 200 displaying the color reference image 210 in an environment where a desired effect is achieved. Here, the desired effect when the correction parameter is to be calculated in advance is the same as the environmental conditions such as illumination that are performed on the re-shooting monitor 200 during the actual shooting by the actual video camera 300 thereafter. It is preferable to do.

(Step S540)
The color correction apparatus 100 captures in the frame buffer 140 a recapture monitor shooting signal of the video camera 300 that has shot the recapture monitor 200 displaying the color reference image 210.

(Step S550)
The multiple model calculation unit 131 based on the multi-constraint FNS method or the multi-constraint extended FNS method of the comparison calculation unit 130 calculates a plurality of color correction models. The color correction model differs depending on the number of parameters h (maximum of 12) indicating the degree of freedom in equation (13) described later.

(Step S560)
The color correction model selection unit 132 selects the color correction model having the minimum value as the optimum one in accordance with the criterion calculated by the selection criterion calculation unit 133. In general, it is known that the color correction model becomes a more precise and fitted model as the number of parameters is increased to increase the amount of calculation. On the other hand, since the calculation load increases as the number of parameters increases, there is a trade-off relationship in which the calculation processing time, power consumption, and heat generation amount increase.

  However, the color correction model selection unit 132 of the color correction apparatus 100 can select a model in which the number of parameters is not excessively increased and the balance with the fitting is balanced according to the criterion calculated by the selection criterion calculation unit 133. .

(Step S570)
The correction parameter calculation unit 134 calculates a correction parameter using the color correction model selected in step S560. Typically, the correction parameter calculation unit 134 can calculate a white level, a black level, and a gamma level for each of RGB.

(Step S580)
The correction parameter storage unit 110 of the color correction apparatus 100 stores the correction parameter calculated in step S570. When there are a plurality of re-imaging monitors 200, correction parameters may be calculated and stored corresponding to each re-imaging monitor. Steps S510 to S580 described above are preparation steps for performing actual correction processing on the input video. Note that after step S580, the verification stage already described in FIG. 2 may be included.

(Step S590)
It is determined whether or not there is an input to the color correction apparatus 100 such as a VTR image or a captured image to be displayed on the re-imaging monitor 200. If there is an input of a VTR video or a video to be displayed on the re-shooting monitor 200, the process proceeds to step S5a0. If there is no input of a VTR video or a video to be displayed on the re-shooting monitor 200, the process stands by in step S590.

(Step S5a0)
The correction processing unit 120 corrects the input video using the correction parameters stored in the correction parameter storage unit 110.

(Step S5b0)
The re-imaging monitor 200 displays the corrected input video 230.

(Step S5c0)
The video camera 300 captures the re-imaging monitor 200 displaying the corrected input video 230. Note that it is preferable that the video camera 300 itself has already been subjected to color tone adjustment.

(Step S5d0)
The video acquired by the video camera 300 is displayed on the master monitor 400. In the image displayed on the master monitor 400, there is a recapture monitor 200 displaying the corrected input video 230 (that is, the recapture monitor input video 420).

  Since the corrected input video 230 has been previously corrected by the color correction device 100 for the effects and display characteristics of the recapture monitor 200, a newscaster or the like photographed within the same angle as the recapture monitor 200 by the video camera 300 or the like. Will be displayed on the master monitor 400 as natural colors equivalent to the subject.

  Next, a part of the paper by the present inventor regarding the color correction process performed by the color correction apparatus 100 will be described with reference to the correction process.

  The color adjustment between multiple monitors is automated for the effect called “re-shooting” that is often performed in television stations. A plurality of color levels are automatically detected from an image obtained by photographing a color chart having known color levels, and color correction is performed so that they become appropriate color levels. The color level of the color chart in the video photographed by the camera may include an observation error and a gamut error due to restrictions on the level of the color space. Even in such a case, an optimum color correction parameter is estimated.

  In TV stations, a monitor is placed in the studio set, and various videos in accordance with the program title and program content are displayed in it. ing.

  The problem at this time is that the color of the image in the monitor does not look correct due to the difference in the color temperature between the studio lighting and the re-shooting monitor, and the color of the monitor is frequently adjusted to match the color temperature. Is called. In the case where there are a plurality of re-imaging monitors, the colors must be matched between the monitors. The same applies to color adjustment between a plurality of cameras used for photographing.

  In this paper, in order to automate the color adjustment work between multiple monitors or cameras, an image of a color chart (color chart) composed of known color levels such as a Macbeth chart (Macbeth in the photo and video industry). It is also called a chart and is used almost standardly. In the case of a camera, an actual color chart is taken. In the case of a monitor, the color chart is displayed on the monitor and taken by the camera. The plurality of color levels are automatically detected, and color correction is performed so that they become appropriate color levels. The color level of the color chart in the video photographed by the camera may include an observation error and a gamut error due to restrictions on the level of the color space. Even in such a case, an optimum color correction parameter is estimated.

(Optimal estimation of color correction parameters)
Although there are many studies on color correction, the influence of gamut errors due to limitations on observation errors and color space levels included in images is not fully considered in the estimation of color correction parameters. For estimation of the color correction model, a least square method is used, or for the color correction model, a higher-order polynomial model or an explicit model is unknown and a neural network is used.

In the RGB three-dimensional color space, consider the following color conversion model based on multiplication of a 3 × 3 matrix called a color matrix and addition of a three-dimensional offset vector (see FIG. 9).

here,

であり、その出力（カラーチャートを撮影した観測画像における色データ）

には、正規分布に従う誤差

が含まれると仮定する。 Input is true color data in color chart

And its output (color data in the observed image of the color chart)

The error according to the normal distribution

Is included.

が与えられたときの尤度は次のようになる。

This is a so-called regression problem (“output error model”), and N sets of color correspondence data

The likelihood when is given is as follows.

の内積を

と書く。

とすると（一様等方性）、結局

However, vector

The inner product of

Write.

(Uniform isotropic)

および

が色変換モデルのパラメータおよび残差分散の最尤推定量を与える。 Maximize this likelihood

and

Gives the maximum likelihood estimator of the parameters and residual variance of the color transformation model.

が与えられたとき、目的変数

の確率分布

が平均

、共分散行列

の（多次元）正規分布で与えられると仮定している。 This is an explanatory variable

Is given, the objective variable

Probability distribution

Is average

, Covariance matrix

Is assumed to be given by a (multidimensional) normal distribution.

の自然対数をとれば、対数尤度

が得られる。最尤推定量を求めるためには、対数尤度

を最大にする

および

を求めればよい。この対数尤度を最大にする

を求めるためには、

を最小にする

を求めればよいことがわかる。すなわち、色変換モデルの場合、最尤法は最小二乗法に帰着する。 Likelihood function

Taking the natural logarithm of, the log likelihood

Is obtained. To find the maximum likelihood estimator, log likelihood

Maximize

and

You can ask for. Maximize this log-likelihood

To ask for

Minimize

It can be seen that That is, in the case of a color conversion model, the maximum likelihood method results in a least square method.

を最小にするパラメータを推定することを考える。ここで、［数６］と［数７］の

は同じものではない。その意味も“変換”とその逆変換である“補正”であることには注意が必要である。 However, what is actually required is not a color conversion parameter, but a color correction parameter that is "inverse conversion" thereof. Therefore, the parameters of the color correction model are estimated by switching the input and output. Therefore,

Consider estimating a parameter that minimizes. Here, [Equation 6] and [Equation 7]

Are not the same. It should be noted that the meaning is “conversion” and “correction” which is the inverse conversion.

  This is an “input error model” in which an error is included in the input, but is expressed in a quadratic form of parameters, so that the solution can be estimated as a unit eigenvector for the minimum eigenvalue of the moment matrix. However, it is known that a statistical deviation occurs in the obtained solution. Kanaya generally formulates the problem of [Equation 7] as a “geometric fit” and proposes a method for optimally estimating it.

と変数

の複数の実現値

からパラメータ

を推定する問題である。誤差は正規分布に従い、各

の正規化共分散行列

（一般には特異行列）は既知とする。この問題の「最尤推定」はマハラノビス距離

を拘束式

のもとで未知数

に関して最小化することである。その残差（

の最小値）を

とし、ノイズレベルを

とすると

は第１近似において自由度

の

分布に従う。ただし、

は拘束条件（［数Ａ１］のランク（独立なものの数）であり、

は未知数

の自由度である。これからノイズレベル

の次の不偏推定量が得られる。

That is, for geometric fitting and geometric AIC / geometric MDL, “geometric fitting” is “constrained”

And variables

Multiple realizations of

To parameters

It is a problem to estimate. The error follows a normal distribution and

Normalized covariance matrix of

(Generally, the singular matrix) is assumed to be known. The "maximum likelihood estimation" for this problem is the Mahalanobis distance

The restraint type

Unknown under

Is to minimize. Its residual (

Minimum value)

And the noise level

If

Is the degree of freedom in the first approximation

of

Follow the distribution. However,

Is the constraint condition ([number A1] rank (number of independent ones),

Is unknown

Is the degree of freedom. Noise level from now on

The next unbiased estimator is obtained.

の空間）に

次元モデル（多様体）

を定義するとき、その幾何学的ＡＩＣおよび幾何学的ＭＤＬは次のように定義される。

The constraint equation [number A1] is the data space (variable

Space)

Dimensional model (manifold)

The geometric AIC and geometric MDL are defined as follows:

はある基準長であり、データ

が

となるように選ぶ。ただし、ノイズレベルは一般モデルにより［数Ａ３］で推定する。 here

Is a reference length and data

But

Choose to be. However, the noise level is estimated by [Equation A3] by a general model.

  Here, the optimal calculation of color correction parameters by three-dimensional affine transformation is based on the optimal estimation of the projective transformation matrix of a two-dimensional plane by the multi-constraint FNS method of Niitsu-Kanaya, which corresponds to the FNS method of Chojnacki et al. Is formulated.

をスケール定数

によって各成分のオーダーをそろえた４次元同次ベクトルを改めて

で表し、３次元アフィン変換行列を

行列

で表す。（色データの量子化ビット数８ビットのため

とした。）

(3D affine transformation)
The color conversion of equation (1) is a three-dimensional affine transformation, and the corresponding color data in the RGB three-dimensional color space

The scale constant

A new four-dimensional homogeneous vector with the order of each component

And the three-dimensional affine transformation matrix

line; queue; procession; parade

Represented by (Because the number of quantization bits of color data is 8 bits

It was. )

Using these, the three-dimensional affine transformation can be written as follows.

は４次元ベクトルの第４成分を１とする正規化作用素である。行列

には定数倍の不定性がある。そこで以下、適当な定数を掛けて

と正規化する（行列のノルムは

と定義する）。［数９］を各成分毎に書くと、

However,

Is a normalization operator in which the fourth component of the four-dimensional vector is 1. line; queue; procession; parade

Has a constant multiple indefiniteness. So, below, multiply by an appropriate constant.

(The matrix norm is

Defined). When [Equation 9] is written for each component,

を次のように定義する。

13-dimensional vector

Is defined as follows.

は［数１３］の

が単位ベクトルであること（

）と等価である。３次元アフィン変換の各成分毎の［数１０］［数１１］［数１２］から次の拘束式を得る。

（３次元アフィン変換の最適計算） Normalization

Of [Equation 13]

Is a unit vector (

Is equivalent to The following constraint equation is obtained from [Equation 10] [Equation 11] [Equation 12] for each component of the three-dimensional affine transformation.

(Optimum calculation of 3D affine transformation)

個の色データ

と対応する色データ

が与えられたとき、それらの間の３次元アフィン変換を最適に計算したい。データ

を代入した［数１４］［数１５］［数１６］のベクトル

をそれぞれ

と書く。データに誤差があるとき、解くべき問題は

となるような１３次元ベクトル

を計算することである。そのためには誤差の統計的性質を考慮する必要がある。各色データ

は真値

に期待値０、標準偏差

の正規分布に従う誤差が加わると仮定する。最尤推定は次の目的関数

を最小にする

を求めることである。

Color data

And corresponding color data

I want to optimally calculate the 3D affine transformation between them. data

[Expression 14] [Expression 15] [Expression 16]

Each

Write. When there is an error in the data, the problem to be solved is

13-dimensional vector such that

Is to calculate For that purpose, it is necessary to consider the statistical nature of the error. Each color data

Is a true value

Expected value 0, standard deviation

It is assumed that an error according to the normal distribution of is added. Maximum likelihood estimation is the objective function

Minimize

Is to seek.

は

を

要素とする

行列の（ムーア・ペンローズの）一般逆行列の

要素である。

However,

Is

The

Element

Of the general inverse of the matrix (Moore Penrose)

Is an element.

は

の正規化共分散行列である。

はデータ

の正規化共分散行列

から線形近似によって次のように計算できる。

here,

Is

Is the normalized covariance matrix.

Is the data

Normalized covariance matrix of

Can be calculated by linear approximation as follows.

は

のヤコビ行列（各成分の

に関する微分を並べた行列）に

を代入した値である（詳細省略）。 However,

Is

The Jacobian matrix of each component

A matrix with derivatives of

Is a value substituted by (details omitted).

の正規化共分散行列

は、再撮モニタを色補正する場合には、モニタに誤差のないカラーチャートを表示したものをカメラで撮影するため、

とする。 ● Data

Normalized covariance matrix of

When recalibrating a re-shoot monitor, the camera displays a color chart with no error on the monitor.

And

とする。

は６次の単位正方行列である。
（多拘束ＦＮＳ法）
［数１８］を

で微分すると次のようになる。

● When correcting the color of the camera, in order to shoot the actual color chart with the camera,

And

Is a 6th order unit square matrix.
(Multiple restraint FNS method)
[Equation 18]

Differentiated by, it becomes as follows.

行列

を次のように置いた。

However,

line; queue; procession; parade

Was placed as follows.

は次のように定義する。

［数１８］を最小にするには［数２１］より

を解けばよく、固有値問題を反復して解くＣｈｏｊｎａｃｋｉらのＦＮＳ法を用いることができる。多拘束ＦＮＳ法の手順は次のようになる。 In the above formula

Is defined as follows.

From [Equation 21] to minimize [Equation 18]

And the FNS method of Chojnacki et al. That iteratively solves the eigenvalue problem can be used. The procedure of the multi-constraint FNS method is as follows.

の初期値

を与える。
２．［数２２］の

を計算する。
３．固有値問題

の最小固有値

に対する単位固有値ベクトル

を計算する。
４．符号を除いて

であれば次のアフィン変換行列

を返して終了する。そうでなければ

としてステップ２に戻る（

は単位ベクトルへの正規化）。ここで、解の更新に

を用いるのは、“中点”

を用いることを意味しており、これは反復の収束を行うための工夫である。 1.

Initial value of

give.
2. [Equation 22]

Calculate
3. Eigenvalue problem

Minimum eigenvalue of

Unit eigenvalue vector for

Calculate
4). Excluding the sign

Then the affine transformation matrix

Return and exit. Otherwise

Return to step 2 as

Is normalized to a unit vector). Here, to update the solution

Uses “midpoint”

This is a device for performing convergence of iteration.

（クロネッカのデルタ：

で１、それ以外で０）と置くと次のようになる。

(Least square method)
The iteration requires an initial value, and the simplest is the least square method. In [Equation 18]

(Kronecca Delta:

If it is set to 1 and 0 otherwise, it becomes as follows.

This is transformed as follows.

［数２５］は

の２次形式であるから、これを最小化する単位ベクトル

はモーメント行列

の最小固有値に対する単位固有ベクトルである。
（精度の評価と理論限界） However, it was placed as follows.

[Equation 25] is

A unit vector that minimizes the quadratic form of

Is the moment matrix

Is the unit eigenvector for the smallest eigenvalue of.
(Accuracy evaluation and theoretical limits)

を表す［数１３］の１３次元単位ベクトル

の推定値を

、誤差がない場合の真値を

とするとき、誤差

を次のように定義する。

Affine transformation matrix

13-dimensional unit vector of [Equation 13] representing

Estimate of

The true value when there is no error

Error when

Is defined as follows.

は

に直交する方向に射影した成分を得る射影行列である。

は１３次の単位正方行列である。これから推定値

の共分散行列が次のように定義できる。

Is

This is a projection matrix for obtaining a component projected in a direction orthogonal to.

Is a 13th order unit square matrix. Estimated value

Can be defined as follows.

を推定しても、それが不偏推定量であればその共分散行列

には次の下界があることが示される。

According to Kanaya's statistical optimization theory, how

If it is an unbiased estimator, its covariance matrix

Indicates that there is the following lower bound.

は左辺引く右辺が正値対称行列であることを示す。右辺の

は［数２２］の行列

を色データの真値

の真値

を用いて計算した値である。また

はランク１２の（ムーア・ペンローズの）一般逆行列を表す。（

対称行列

のランク

の一般逆行列

とは、

の大きい

個の固有値を逆数に置き換え、残りの固有値を０に置き換えた行列である。） However,

Indicates that the right side minus the left side is a positive symmetric matrix. Right side

Is a matrix of [Equation 22]

The true value of the color data

True value of

This is a value calculated using. Also

Represents the general inverse matrix of rank 12 (Moore Penrose). (

Symmetric matrix

Rank

General inverse of

Is

Big of

It is a matrix in which the eigenvalues are replaced with reciprocals and the remaining eigenvalues are replaced with 0. )

であることに注意すると、次の平方平均二乗（ＲＭＳ）誤差の下界が得られる。

The right side of [Equation 29] is called the “KCR lower bound”. Taking the square root of both sides of [Equation 29], from [Equation 28]

Note that a lower bound on the following root mean square (RMS) error is obtained.

(Level-constrained optimal estimation and model selection)
Since it is a color conversion process in the color space, a clipping process, which is strictly a limitation on the level, is attached (see [FIG. 10]).

That is, the color conversion by the three-dimensional affine transformation of [Equation 1] is as follows.

であり、

は色空間における最大レベルである。

は色データをＲＧＢ３次元色空間における３次元ベクトルと見なして、そのベクトルの方向を保存する様に各色成分を各上限下限成分にクリップする処理である（詳細省略）。 here,

And

Is the maximum level in color space.

Is a process of regarding color data as a three-dimensional vector in the RGB three-dimensional color space and clipping each color component to each upper and lower limit component so as to preserve the direction of the vector (details omitted).

  Gamut errors caused by clipping, which is a limitation on the level of color space, affect the estimation of color correction parameters.

  In the estimation of color correction parameters, a simple method that avoids the effects of clip processing is a method that does not use color data in the vicinity of the upper and lower limits of the color space. Color correction results cannot be obtained. It is impossible to restore details near the upper and lower limits of the clipped color space. However, it is obviously not desirable that the color correction results at the upper and lower color levels are different.

  Therefore, in consideration of the fact that the observed image includes a gamut error, the color correction parameter is estimated by optimization with level restriction using the correspondence of the color data in the upper and lower limits of the color space as a constraint condition. As an optimization method, an extended FNS method that is an extension of the FNS method so as to obtain a solution that automatically satisfies internal constraints between variables is used.

は、

次元ベクトルであり、それらの真の値

は次の拘束式を満たすとする。

(Internal restraint)
data

Is

Dimensional vectors, their true values

Suppose that the following constraint is satisfied.

は未知の

次元ベクトルである。問題は誤差のあるデータ

から

を推定することである。 However,

Is unknown

It is a dimension vector. The problem is erroneous data

From

Is to estimate.

と正規化するが、これも一つの内部拘束である。そして、これ以外に

に次の

の内部拘束が存在するとする。
［数３３］

Since the scale of the unknown vector is indefinite in the form of [Equation 32],

This is also an internal constraint. And besides this

To the next

Suppose that there are internal constraints.
[Equation 33]

個の式は代数的に独立であるとする。最尤推定量

は

these

Let each expression be algebraically independent. Maximum likelihood estimator

Is

のもとで最小化するものである。 The internal restraint

Minimize under.

here,

の初期解を与える。
２．次の行列

を計算する。

ここで、

３．ベクトル

を計算し、これにシュミットの直交化を施した正規直交系

を求める。
４．次の射影行列

を計算する。

ここで、

は

次の単位正方行列である。
５．次の行列

を計算する。

６．固有値問題

の絶対値の小さい

個の固有値に対する単位固有ベクトル

を求める。
７．現在の解

を次のように

に射影した

を計算する。

８．次の

を計算する。

９．

なら

を返して終了する。そうでなければ、

としてステップ２（行列

の計算）に戻る。 (Multi-constrained extended FNS method)
The procedure of the multi-constrained extended FNS method is as follows.
1.

Gives the initial solution.
2. Next matrix

Calculate

here,

3. vector

And an orthonormal system with Schmidt orthogonalization

Ask for.
4). Next projection matrix

Calculate

here,

Is

The next unit square matrix.
5). Next matrix

Calculate

6). Eigenvalue problem

Small absolute value of

Unit eigenvectors for the eigenvalues

Ask for.
7). Current solution

As follows

Projected onto

Calculate

8). next

Calculate

9.

Then

Return and exit. Otherwise,

As step 2 (matrix

Return to (Calculation).

(Select color correction model)
If a gamut error has occurred, even if it is linear color conversion, in order to obtain a sufficient color correction result, for example, if a color correction model that is the inverse conversion is a polynomial model, a higher-order term is Need to be included. However, whether or not a gamut error is included is unknown, and when a gamut error is included, the color correction model is also unknown. The order of the color correction model is also limited, and color correction using a color correction model including an excessive high-order term is generally unstable. Therefore, it is considered to select a realistic color correction model by selecting a model from the result of fitting a plurality of color correction models.

  Typical model selection criteria in “geometric fitting” include “geometric AIC” and “geometric MDL”. Kanazawa and Kanaya used geometric model selection by geometric AIC to generate image mosaic. Matsunaga and Kanaya used geometric model selection with geometric AIC and geometric MDL for calibration of mobile cameras. Kanatani used geometric AIC and geometric MDL to separate multiple moving objects.

の当てはめ残差

から二乗ノイズレベルを次のように推定する。 If the three-dimensional affine transformation model is a general model having the highest degree of freedom, the degree of freedom is 12, and the objective function is the result of optimal estimation of the parameters.

Fit residual

To estimate the square noise level as follows:

と［数４３］より推定した二乗ノイズレベル

から次のように計算される。

The geometrical AIC and geometrical MDL of the three-dimensional affine transformation model are the residuals of fitting the objective function as a result of optimal parameter estimation.

And square noise level estimated from [Equation 43]

Is calculated as follows.

は基準長であり、色データの量子化ビット数８ビットの場合２５５とする。３次元アフィン変換のランクは３、補正モデルの次元は３である。モデルの次元の項は各モデルに共通の場合には除外してもよい。 here

Is the reference length, and is 255 when the number of quantization bits of the color data is 8 bits. The rank of the three-dimensional affine transformation is 3, and the dimension of the correction model is 3. Model dimension terms may be excluded if they are common to each model.

と［数４３］より推定した一般モデルの二乗ノイズレベル

から計算することができる。幾何学的ＡＩＣあるいは幾何学的ＭＤＬが最小になるモデルを選択すればよい。 Similarly for other models, the residual of fitting the objective function as a result of optimal parameter estimation

And the square noise level of the general model estimated from [Equation 43]

Can be calculated from A model that minimizes geometric AIC or geometric MDL may be selected.

は次のようになる。 (Experiment)
(Numerical simulation of optimal estimation of color correction parameters by three-dimensional affine transformation)
A matrix that corrects the observed image to the color level of the reference image by three-dimensional affine transformation so that it becomes a reference image based on an ideal color chart

Is as follows.

として、対応する観測画像の色データ

に独立に期待値０、標準偏差

の正規乱数誤差を加えた。ただし、色データに対してはクリップ処理は行わなかった。そして計算したアフィン変換行列

を［数１３］の単位ベクトル

の形に表し、その誤差を［数２７］の

とする。これを各

で誤差を変えて１０００回試行し、そのＲＭＳ誤差を次のように評価した。

Data on true color level of color chart

As the color data of the corresponding observation image

Expected value 0, standard deviation independently

The normal random error of was added. However, the clipping process was not performed on the color data. And the calculated affine transformation matrix

Is the unit vector of [Equation 13]

The error is expressed in [Formula 27].

And Each

The error was changed 1000 to try 1000 times, and the RMS error was evaluated as follows.

は誤差

の

回目の試行の値である。［図６］（ｃ）は横軸に

をとって最小二乗法、多拘束ＦＮＳ法のＲＭＳ誤差ＥをＫＣＲ下界（［数３０］の右辺）とともにプロットしたものである。これから分かるように最小二乗法に対して多拘束ＦＮＳ法は、色データがクリップ処理されていない場合においては、精度の理論限界にほぼ到達している。

Is the error

of

The value of the first attempt. [Fig. 6] (c) is on the horizontal axis.

The RMS error E of the least square method and the multi-constrained FNS method is plotted together with the KCR lower bound (the right side of [Equation 30]). As can be seen from this, the multi-constrained FNS method is almost reaching the theoretical limit of accuracy when the color data is not clipped with respect to the least square method.

(Optimal estimation and model selection with level constraints for color correction parameters using a polynomial model)
[FIG. 7] is a simple example when linear conversion is performed as color conversion on a one-dimensional ideal ramp image. In order to estimate the color correction parameter, a case where only uncolored color data is used is compared with a case where level data is imposed with non-clipped color data.

The polynomial model is optimally estimated by the extended FNS method. In the figure, the color data used for the calculation is indicated by “◇ Data”. For the color correction model, a first-order to fourth-order polynomial is used as a polynomial model. Since the optimal estimation of the color correction parameter by the one-dimensional polynomial model is one constraint equation, the next objective function J may be minimized with the white / black levels as constraint conditions. By imposing white and black level constraints, the degree of freedom of the polynomial model is reduced by two. In the case of the primary model, since the parameters are uniquely determined if two points are set as level constraints, the constraint condition is set only for the white level.

であり、ある特定の色レベル

が特定の色レベル

に補正されるとすると、色レベルに関する内部拘束は次のようになる。

For example, in the case of correction by a second order polynomial model,

A certain color level

Is a specific color level

Is corrected, the internal constraint on the color level is as follows.

  In the result of color correction using only the color data that is not clipped, the level of the clipped color data is not correctly corrected [(c) in the figure]. By using the white / black level as a constraint, the white / black level is strictly corrected [(f)]. As a result of model selection, a fourth-order polynomial model was selected by geometric AIC, and a third-order polynomial model was selected by geometric MDL. Geometric MDL tends to choose a degenerate model.

(Image processing for color chart detection)
The color level of the color chart is automatically detected from the image obtained by photographing the color chart by image processing. Each image for each RGB component is converted into a plurality of binary images by a plurality of threshold values. Rectangular regions in the plurality of binary images are extracted by boundary tracking processing. Whether or not the obtained rectangular area is a part of the color chart is determined from the size of the rectangular area thus obtained, the angle formed by the four vertices, and the average and variance of the pixel values in the rectangular area. By sorting in the two directions according to the upper left vertex coordinates of the rectangular area thus selected, a color chart composed of 24 colors is identified as the entire rectangular area. The average value of pixels in a certain area from the barycentric coordinates of each rectangular area is used as the color level in that area.

がカラーチャート中の特定の色レベル

に色補正されるとする。３次元アフィン変換により色補正される場合には、

が内部拘束になる。 (Real image experiment (color correction of re-shoot monitor video))
The color level of the color chart is automatically detected by image processing. A color correction parameter for aligning with the color level of the ideal color chart is optimally estimated, and the captured image is color corrected. Specific color level in the observed image

Is a specific color level in the color chart

It is assumed that color correction is performed. When color correction is performed by three-dimensional affine transformation,

Becomes internal restraint.

  With the white level in the color chart as a constraint condition, a color correction result using a color correction parameter optimally estimated with a level constraint by the multi-constrained extended FNS method is obtained. Here, not only the monitor image but also the entire captured image is color-corrected. (Actually, color correction is performed only on the image projected on the monitor.)

  The RMS error of the color level of the color chart is 27.7, and the white balance is strictly taken. The RMS error when optimally estimated by the multi-constraint FNS method without imposing a level constraint is 19.3, which is slightly higher than that, but lower than 32.7 when the least square is estimated.

(Summary)
In order to perform color correction of the re-imaging monitor, a color chart having a known color level displayed on the monitor was photographed with a camera, and the color level of the color chart in the photographed image was automatically detected. Color correction was performed by optimal parameter estimation considering observation error and gamut error.

  As a future problem, it is also necessary to examine color correction by a color correction model in a luminance color difference color space that is typically used in video signals. As the color correction model, a three-dimensional affine transformation model and a one-dimensional polynomial model are used, but other models may be considered. As a specific application example, application to stereoscopic video shooting is also conceivable.

  FIG. 8 is a schematic diagram for explaining a case where a standard known color chart 8210 such as a Macbeth chart is photographed by a plurality of video cameras 8300 (1) and 8300 (2) and corrected using the color correction device 8100. It is. The color chart 8210 is composed of squares indicating 24 different colors of 4 × 6.

  The color correction apparatus 8100 verifies whether or not the hue of each video image captured by the plurality of video cameras 8300 (1) and 8300 (2) is acquired as a predetermined hue corresponding to the color chart 8210. After correcting the color to a predetermined color corresponding to the chart 8210, the video is output to the video switcher 8900.

  The present invention can be widely used for broadcasting and industrial video camera systems.

  100..Color correction device, 200..Re-shoot monitor..300..Video camera, 1000..Video camera system.

Claims (6)

A reference storage unit for storing a color reference signal;
A buffer to which a recapture monitor photographing signal obtained by photographing a recapture monitor that displays a color reference image corresponding to the color reference signal;
The re-shooting monitor shooting signal stored in the buffer and the color reference signal stored in the reference storage unit are compared, and the correction parameter is set so as to reduce the difference between the re-shooting monitor shooting signal and the color reference signal. A comparison operation unit for calculating
A correction parameter storage unit for storing the calculated correction parameter;
A color correction apparatus comprising: a correction processing unit that performs correction processing based on the correction parameter stored in the correction parameter storage unit for input video data. The color correction apparatus according to claim 1,
The color correction apparatus, wherein the color reference image corresponding to the color reference signal is displayed on the re-shooting monitor by reading out the color reference signal from the reference storage unit and outputting it to the re-shooting monitor. The color correction apparatus according to claim 1 or 2,
The correction processing unit reads the color reference signal stored in the reference storage unit, performs correction processing based on the correction parameter for the color reference signal, and outputs a verification signal to the re-shooting monitor. ,
The buffer is inputted with a re-shooting monitor shooting verification signal obtained by shooting the re-shooting monitor displaying the verification signal,
The comparison operation unit compares the color reference signal stored in the reference storage unit with the re-shooting monitor shooting verification signal stored in the buffer, and verifies whether there is a difference. Correction device. A color correction device according to any one of claims 1 to 3,
A re-shooting monitor that displays the color reference image corresponding to the color reference signal;
A video camera that outputs the recapture monitor photographing signal obtained by photographing the recapture monitor to the color correction device;
A video camera system comprising: A color correction method for correcting a hue of an image obtained by photographing a re-shooting monitor using the color correction device according to any one of claims 1 to 3,
Displaying a color reference image corresponding to the color reference signal on the re-shooting monitor;
Obtaining a recapture monitor shooting signal from a video camera that has captured the color reference image displayed on the recapture monitor;
A step of calculating a correction parameter by comparing the acquired re-shooting monitor photographing signal and a color reference signal stored in the reference storage unit;
Performing a correction process on a video signal input using the calculated correction parameter. A color correction method, comprising: The color correction method according to claim 5,
In the step of displaying the color reference image corresponding to the color reference signal on the re-shooting monitor, the color correction device outputs the color reference signal stored in the reference storage unit, and the color reference image based on the output is reproduced again. A color correction method characterized by being a step of displaying on an imaging monitor.

Images