JP2013083872A - Projection luminance adjustment method, projection luminance adjustment device, computer program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily acquire a color mixing matrix for linearly approximating output luminance from a projector to observation luminance by a camera, regardless of output performance of the projector, camera sensitivity or dynamic range.SOLUTION: An image input part 2 acquires image data from a camera 20 having captured an image projected on a screen. A luminance change detection part 3 performs projective transformation to map pixels on the image onto pixels on a sample image, creates a projective transformation image, and detects luminance change, of images observed one after another, from the sample image. A luminance update part 4 updates luminance to be outputted from a projector 10, according to the luminance change, and projects the image onto the screen from the projector 10. A correction coefficient calculation part 5 calculates a parameter of luminance correction when an image close to a certain target image is observed.

Description

  The present invention provides a projection brightness adjustment technique for automatically correcting the output brightness value from a projector so that the observed brightness value of each pixel becomes a predetermined value when observing a projection image from one or more projectors with a camera. About.

  Projector / camera systems are used in various applications. For example, in addition to using a projector for a conference or presentation at an academic conference presentation, it can also be used as a display system for human interaction or augmented reality. Furthermore, if a plurality of projectors are used, images from the projectors can be connected to generate a huge image (tiling), and in recent years, it is also expected as an application for displaying HDR (High Dynamic Range) images.

When an image from a projector is projected and the projection image is observed by the camera, the original image is adjusted so that the camera observes a predetermined luminance. This is called luminance correction, and is disclosed in Non-Patent Documents 1 and 2, for example. In this method, attention is paid to the correlation between the RGB luminance values from the projector and the RGB luminance values observed by the camera, and a predetermined 3 × 3 matrix V called a color mixing matrix is used. The RGB output value P = (P _R , P _G , P _B ) from the projector is corrected so that the camera observes the RGB luminance value C = (C _R , C _G , C _B ). In this system, the following expression (1) is established between the luminance value C of the observed image and the output luminance P from each projector.

V is a color mixing matrix of the projector, and F = (F _r , F _g , F _b ) is ambient light including illumination from a light source other than the projector (considering the reflectance of the object surface is constant). According to Non-Patent Document 2, if four sample images can be prepared, a color mixing matrix and ambient light can be obtained.

S. K. Nayar, H. Peri, M. D. Grossberg, and P. N. Belhumeur: “A Projection System with Radiometric Compensation for Screen Imperfections”, Proc. Of ICCV Workshop on Projector-Camera Systems (PROCAMS), 2003. K. Fujii, MD Grossberg, and SK Nayar: “A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments”, Proc. Of IEEE Computer Vision and Pattern Recognition (CVPR), vol.1, pp.814-821 , 2005.

As described above, according to Non-Patent Document 2, four images are required to calculate the color mixing matrix between the projector and the camera. As an example of this image, four images of a gray image, a red image, a green image, and a blue image are used (hereinafter referred to as sample images). For example, the RGB values of all the pixels of the gray image are set to P _w = (R _w , G _w , B _w ), and all the pixel values of the red image are set to P _r = (R _w + ΔR, G _w , B _w ). , All pixel values of the green image are set to P _g = (R _w , G _w + ΔG, B _w ), and all pixel values of the blue image are set to P _b = (R _w , G _w , B _w + ΔB). . In other words, with a gray image as a reference image, a red image, a green image, and a blue image are images in which only one channel is changed.

Non-Patent Document 2 uses these images to easily detect the luminance change of each pixel and calculate the elements of the color mixing matrix. For example, when a gray image and a red image are output from a projector and observed with a camera, certain pixel values are C _w = (R _w ′, G _w ′, B _w ′), _Let C _r = (R _r ′, G _r ′, B _r ′). Since the projector changes in luminance only in the R channel, according to the equation (1), the relationship of the following equation (2) is obtained from the difference C _r −C _w .

Therefore, the elements V _rr , V _gr , and V _br of the color mixing matrix can be obtained by the calculation of the following equation (3). Similarly, the elements of the remaining color mixing matrix can be estimated by using the green image and the blue image.

  As is apparent from the equation (1), the conventional method is a method of linearly approximating the luminance P from the projector and the observation luminance C by the camera. Therefore, if the change amount of each channel is too small, the equation (3 ) May degrade the accuracy of each matrix element (being inversely proportional to the amount of change).

  In other words, in order to appropriately provide each change amount ΔR, ΔG, and ΔB of the sample image, the operator stabilizes each element of the color mixing matrix in consideration of the output performance of the projector, camera sensitivity, dynamic range, and the like. And had to be estimated. In other words, there is a problem that the color mixing matrix cannot be automatically estimated without considering the output performance of the projector, camera sensitivity, dynamic range, and the like.

  The present invention has been made in consideration of such circumstances, and its purpose is to output brightness from the projector and the camera based on an arbitrary sample image regardless of the output performance, camera sensitivity, and dynamic range of the projector. It is an object of the present invention to provide a technique capable of easily obtaining a color mixing matrix for linearly approximating between the observed luminances obtained from the above.

  One aspect of the present invention is a projection brightness adjustment method in a projector / camera system that captures an image projected by a projector using a camera, and a projection image obtained by projecting a sample image based on a set value by the projector, An observation image acquisition step acquired as an observation image observed by the camera, a luminance change detection step of detecting a luminance change between the sample image and the observation image, and preset based on the performance of the projector or the camera And a brightness control step for controlling the brightness of the projector based on the detected amount of brightness change without using a correction parameter.

  One aspect of the present invention is the above-described projection luminance adjustment method, wherein the observation image acquisition step and the luminance change detection step are repeatedly executed, and the control step includes the amount of the luminance change being a predetermined allowable value or more. In this case, after updating the luminance input to the projector, projecting the image again based on the updated luminance from the projector, and when the amount of luminance change is smaller than the predetermined allowable value And a completion determination step for determining that the control of the brightness of the projector for the sample image has been completed.

  One aspect of the present invention is the projection brightness adjustment method described above, further including a calculation step of calculating a correction parameter including a color mixture matrix and an ambient light vector after the brightness control step.

  One aspect of the present invention is the projection brightness adjustment method described above, wherein a test image for calculating a gradation error is generated, the test image is corrected using the correction parameter, and the corrected image is corrected. Calculating a gradation error representing a luminance error in each gradation between a test observation image obtained by capturing a projected image of the test image with the camera and the test image before correction; and the gradation error Is equal to or greater than a predetermined threshold, the luminance gradation of the sample image used for calculating the correction parameter is changed, the sample image is corrected again by the luminance control step, and the gradation error is predetermined A step of determining that the correction of the sample image has been completed if it is smaller than the threshold value.

  One aspect of the present invention is a projection luminance adjustment apparatus for a projector / camera system that captures an image projected by a projector using a camera, and a projection image obtained by projecting a sample image based on a set value by the projector An observation image acquisition unit acquired as an observation image observed by a camera, a luminance change detection unit for detecting a luminance change between the sample image and the observation image, and a correction set in advance based on the performance of the projector or the camera And a brightness control unit that controls the brightness of the projector based on the detected amount of brightness change without using a parameter.

  One aspect of the present invention is a computer program that causes a computer to execute the above-described projection luminance adjustment method.

  One aspect of the present invention is a recording medium on which a computer program for causing a computer to execute the above-described projection luminance adjustment method is recorded.

  According to the present invention, even if an arbitrary sample image is given without considering the output performance of the projector, camera sensitivity, dynamic range, etc., the linear approximation is performed between the output luminance from the projector and the observation luminance by the camera. The color mixing matrix can be automatically obtained.

It is a block diagram which shows the structure of the projection luminance adjustment apparatus 1 of 1st Embodiment which concerns on this invention. It is a flowchart for demonstrating the operation | movement of this 1st Embodiment. It is a conceptual diagram which shows the typical example of the characteristic (tone curve) of the luminance output which observed the input luminance (a certain color channel) given to the projector with the camera. It is a conceptual diagram which shows the example of the brightness | luminance correction | amendment using linear approximation (approximation with a narrow range). It is a conceptual diagram which shows the example of the brightness | luminance correction | amendment using linear approximation (approximation with a wide range). It is a block diagram which shows the structure of the projection luminance adjustment apparatus 1 of 2nd Embodiment which concerns on this invention. It is a conceptual diagram for explaining a method of setting the input brightness level P _L and P _H in the second embodiment. It is a flowchart for demonstrating operation | movement of the model evaluation part by this 2nd Embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

A. First Embodiment FIG. 1 is a block diagram showing a configuration of a projection luminance adjusting apparatus 1 according to a first embodiment of the present invention. In FIG. 1, the projection brightness adjusting apparatus 1 realizes a projection brightness adjusting method for a projector / camera system including one projector 10 and one camera 20. The projection luminance adjusting apparatus 1 includes an image input unit 2, a luminance change detection unit 3, a luminance update unit 4, and a correction coefficient calculation unit 5. The image input unit 2 acquires image data from the camera 20. The luminance change detection unit 3 acquires an observation image from the image input unit 1 and detects a luminance change using the image information. The luminance update unit 4 generates a projection conversion image using the plane projection conversion data extracted from the projection calculation DB 6, and outputs the luminance output from the projector 10 according to the luminance change obtained by the luminance change detection unit 3. Update. The correction coefficient calculation unit 5 calculates a parameter for luminance correction at the time when an image close to a certain target image is observed, and stores it in the luminance correction DB 7.

  The projection brightness adjusting apparatus 1 according to the first embodiment updates the brightness given to the projector 10 according to the brightness change of the observed image so that the camera 20 observes an image close to the prepared sample image, and automatically Thus, a luminance correction coefficient for correcting the luminance of an arbitrary image is calculated.

  In this configuration, the projector 10 and the camera 20 do not necessarily have to be connected as constituent elements, and may acquire data necessary for processing. Further, the flow of data from the image input unit 2, the luminance change detection unit 3, the luminance update unit 4, and the correction coefficient calculation unit 5 to the respective arrows uses a recording medium such as a hard disk, a RAID device, or a CD-ROM. Alternatively, either a form using a remote data resource via a network may be used.

  Details of each part will be described below. Note that sample images in the following processing units indicate gray images, red images, green images, and blue images.

(Image input unit 2)
The image input unit 2 acquires an image observed (captured) by the camera 20 and projected onto the screen (plane) by the projector 10 and transfers the image to the luminance change detection unit 3. An example of an image acquired by the image input unit 2 is an image captured without hiding the entire screen, or an image obtained by photographing a predetermined area.

(Brightness change detection unit 3)
The luminance change detection unit 3 performs projective transformation for associating the pixels on the image with the pixels on the sample image, and generates a projected transformed image. The projective transformation method will be described below.

The two-dimensional coordinates of the points on the projector screen are (u, v), the three-dimensional coordinates when the points are projected on the XY plane with Z = 0 (X, Y, 0), and the projected points are captured by the camera 20. Assuming that the two-dimensional coordinates of the observed point are (x, y), the following expression (4) is obtained by the plane projection transformation H _ps between the projector 10 and the plane and the plane projection transformation H _sc between the plane and the camera 20. Each pixel can be associated with each other by the calculation of (5).

  Furthermore, the two-dimensional coordinates (u, v) of the image output from the projector 10 and the two-dimensional coordinates (u ′, v ′) of the points on the sample image are associated with each other by the calculation of the following equation (6). Can do.

H _ip is a planar projective transformation between the sample image and the projector screen. Therefore, if these planar projective transformations are known, the two-dimensional coordinates (u ′, v ′) of the sample image and the two-dimensional coordinates (x, y) observed by the camera 20 and on the projector screen are displayed. The two-dimensional coordinates (u, v) and the two-dimensional coordinates (x, y) observed by the camera 20 can be linked. These plane projective transformations are obtained by a prior calibration work, and the data is stored in the projective transformation DB 6. In this calibration, a known planar projective transformation estimation method can be used.

  When the luminance change detection unit 3 acquires the observation image from the image input unit 2, the luminance change detection unit 3 reads the plane projection transformation data from the projection transformation DB 6, and according to Equation (6), the two-dimensional coordinate values (u ′, v) of the pixels of the sample image The two-dimensional coordinate value (x, y) corresponding to the pixel on the observation image is calculated from '). The luminance change detection unit 3 applies this coordinate calculation to all the target pixels, and generates an image obtained by projective transformation of the observed image.

  Since each pixel of the projective transformation image is associated with each corresponding pixel of the sample image, the pixel value C ′ (t) of the observed image can be compared with the pixel value C on the sample image. Since the images are sequentially observed from the camera 20 until the operation of the first embodiment is completed, the pixel value of the observed image at the state t or at the time t is represented as C ′ (t). The luminance change ΔC (t) of each pixel is calculated by calculation of the following equation (7) while associating each pixel of the original image and the observed image using the planar projective transformation.

  The luminance change detection unit 3 detects a luminance change with respect to the sample image using Equation (7) for the sequentially observed images.

(Luminance update unit 4)
When the luminance update unit 4 is activated, it first initializes an image to be set on the projector 10. For example, the input image given by the following equation (8) for each pixel is set in the projector 10.

  C corresponds to each pixel of the sample image. However, α is a parameter. For example, α = 0.5. In order for the image to be projected onto a predetermined plane or region, plane projection transformation data is extracted from the projection transformation DB 6, and a projection conversion image is generated from the image using the plane projection transformation data. The planar projective transformation is used once read out and temporarily stored in a memory (not shown). The image set in the projector 10 is projected onto a screen (not shown) that is a predetermined plane.

  Assume that the projector input luminance in the state t is P (t). The luminance update unit 4 calculates an average value (luminance error) of errors for all pixels from the luminance change ΔC (t) obtained by the luminance change detection unit 3 described above, and the luminance error is a predetermined allowable value ε (for example, , Set ε = 10), the process is stopped at the present time, and the process proceeds to the correction coefficient calculation unit 5 (the processing content of the correction coefficient calculation unit 5 will be described later). On the other hand, if the luminance error is greater than or equal to the predetermined allowable value ε, the projector input luminance value in state t + 1 is updated by the following equation (9).

  Here, K is a 3 × 3 matrix and plays a role as a feedback gain. In the first embodiment, the following equation (10) is assumed.

  The parameter k may be set freely. For example, in the form of correcting the brightness of one projector 10, k = 1.0 is set. In the form of correcting the brightness of the two projectors 10 at the same time, k = 0.5 is set and In the form of correcting the luminance in the projector 10 at the same time, k = 0.3 is set. The pixel whose luminance has been updated using Expression (9) is set to the pixel on the image given to the projector 10 as the next state. At this time, similarly to the projection conversion of the observation image, the projection conversion image is generated using the plane projection conversion data extracted from the projection calculation DB 6.

  The pixel of the conversion image for projection is a pixel of the projector screen. An image buffer having the same size as the sample image is prepared, and the pixels correspond to (u ′, v ′). Coordinates (u, v) on the projector screen are calculated. The pixel whose luminance has been updated is set at the point of the coordinates (u, v). After updating the luminance of all the pixels, the luminance updating unit 4 sets the projection conversion image on the projector 10 and projects the updated image on the plane again.

  The luminance update according to Equation (9) is performed according to the luminance change in each state t, and is continued until the luminance error becomes smaller than the allowable error ε. For each sample image, when the luminance error becomes smaller than the allowable error ε, it is determined that the luminance correction of each sample image is completed, and is temporarily secured in the working memory.

(Correction coefficient calculation unit 5)
The correction coefficient calculation unit 5 executes the operation when the processing of the luminance update unit 4 is completed for the four sample images. The correction coefficient calculation unit 5 takes out a luminance correction image related to each sample image that has been temporarily stored. For a pixel of interest, in the gray-based image, the pixel value P _W, in the red-based image, the each pixel P _R, in the green-based image, the each pixel P _G, in the blue-based image, the each pixel Let P _B. Since these pixel values are pixels on an image set in the projector, they will be simply referred to as projector input luminance.

On the other hand, as original pixel values, each pixel of the gray image is C _W , each pixel of the red image is C _R , each pixel of the green image is C _G , and each pixel of the blue image is C _B. Since these pixel values are almost equivalent to the pixels on the image obtained by camera observation, they will be simply referred to as camera observation luminance. The relationship between the projector input luminance and the camera observation luminance can be linked by Equation (1) using the color mixing matrix and the ambient light vector.

  When the inverse matrix of the color mixing matrix V is U, the luminance difference between the red image and the gray image is expressed by the following equation (11).

Since the observed image is substantially equal to the sample image, the left-hand side of equation _{(11) (C R -C W} ), R component only without value and the other component is a value 0. That is, Formula (11) becomes the following Formula (12).

However, ΔR is an R component of (C _R −C _W ), and (ΔR _r , ΔG _r , ΔB _r ) is each RGB value of the projector input luminance difference P _R -P _W. Therefore, from Equation (12), the inverse matrix element of the color mixing matrix can be calculated by the following Equation (13).

  Similarly, the following equation (14) is obtained from the difference between the green image and the gray image.

  Similarly, the following equation (15) is obtained from the difference between the blue image and the gray image.

However, (ΔR _g , ΔG _g , ΔB _g ) is an RGB value of the projector input luminance difference (P _G −P _W ), and (ΔR _b , ΔG _b , ΔB _b ) is a projector input luminance difference (P _B- P _W ) RGB values. Thus, since the inverse matrix of the color mixing matrix is obtained, the inverse matrix of the matrix U is calculated to obtain the color mixing matrix V. Furthermore, the ambient light vector F is calculated using the following equation (16) from the relationship of the equation (1) using the gray image.

  The correction coefficient calculation unit 5 obtains the color mixture matrix V and the ambient light vector F for all pixels, and then stores these data in the brightness correction DB 7 (in the present invention, the color mixture matrix, the ambient light vector, Are called correction parameters).

  FIG. 2 is a flowchart for explaining the operation of the first embodiment. In the flowchart shown in FIG. 2, processing is performed in the order of a gray image, a red image, a green image, and a blue image. First, when the operation is started, the luminance update unit 4 sets a sample image (step S1), and determines whether or not the correction is completed (step S2). If the correction has not been completed (NO in step S2), the luminance value of the image to be set on the projector 10 is initialized (step S3), and the image is projected onto a predetermined plane or region. Then, plane projection conversion data is extracted from the projection conversion DB 6, and a projection conversion image is generated from the image using the plane projection conversion data (step S9), and projected from the projector 10 onto a screen that is a predetermined plane (step S9). Step S10).

  The image input unit 2 acquires the image observed (captured) by the camera 20 and projected on the screen (plane) by the projector 10 and transfers it to the luminance change detection unit 3 (step S4). The luminance change detection unit 3 extracts plane projection conversion data from the projection conversion DB 6, performs projection conversion for associating the pixels on the image with the pixels on the sample image, and generates a projection conversion image (step S5). The luminance change detection unit 3 detects a luminance change with the sample image for the sequentially observed images (step S6).

  The luminance update unit 4 calculates an average value of errors (luminance error) for all pixels from the luminance change obtained by the luminance change detection unit 3 described above, and the luminance error is a predetermined allowable value ε (for example, ε = 10). It is judged whether it is smaller (step S7). If the luminance error is equal to or greater than the predetermined allowable value ε (NO in step S7), the luminance update unit 4 updates the projector input luminance value (step S8) and updates the luminance of all pixels. As described above, a projection conversion image is generated using the plane projection transformation data extracted from the projection transformation DB 6 (step S9), and the updated image is projected again from the projector 10 onto a flat screen (step S10). ).

  The processes in steps S4 to S10 described above are executed while updating the brightness until the brightness error becomes smaller than the allowable error ε (YES in step S7). The luminance update unit 4 determines that the luminance correction of each sample image is completed (YES in step S2) when the luminance error is smaller than the allowable error ε for each sample image (YES in step S2). The process proceeds to the coefficient calculation unit 5. The correction coefficient calculation unit 5 obtains the color mixing matrix V and the ambient light vector F for all the pixels, and then stores these data in the luminance correction DB 7 (step S11).

  According to the first embodiment described above, the operator does not need to consider the optical internal parameters of the projector / camera system in advance, and the brightness between the projector input brightness and the camera observation brightness from the appropriately given sample image. The error is automatically corrected, and the color mixing matrix V and the ambient light vector F of each pixel can be obtained with the same accuracy as the conventional method.

  Further, according to the first embodiment, it is possible to take out correction parameters from the luminance correction DB 7 according to the application and correct the luminance of an arbitrary content image according to the conventional method of Non-Patent Document 1 or 2. .

  Note that if the flowchart according to the first embodiment is applied to a plurality of projectors all at once, the brightness of the multi-projector system can be automatically corrected and the respective correction parameters (color mixing matrix and ambient light vector) can be obtained. .

B. Second Embodiment Next, a second embodiment of the present invention will be described.
Before describing the second embodiment, a new problem to be solved by the second embodiment will be described. FIG. 3 is a conceptual diagram showing a typical example of a characteristic (tone curve) of luminance output obtained by observing input luminance (a certain color channel) given to the projector with a camera. 4 and 5 are conceptual diagrams showing examples of luminance correction using linear approximation.

  In general, the sensitivity of the camera is less responsive at the low gradation level of the section A, shows an almost linear characteristic at the intermediate gradation of the section B, and has a gentle slope at the high gradation of the section C. In the luminance correction of the projector / camera system in the conventional method, linear approximation as shown by the thick straight lines a and b shown in FIG. 4 or FIG. The input luminance given to the projector is corrected so as to observe the output luminance.

In FIG. 4, the tone curve is linearly approximated by a straight line a connecting two points of the input luminance levels P _L and P _{H. In} FIG. 5, the straight line b connecting P _L with the input luminance level P _H increased as much as possible. The tone curve is linearly approximated. In Figure 4, the approximate line a relative higher P _H input luminance gradually out of the tone curve, in FIG. 5, which approximates the relatively wide range in a straight line b.

That is, FIG. 5 shows that the correction over a wide range can be controlled because the error from the original tone curve is small over a wide range. The input luminance levels P _L and P _H correspond to the luminances of the gray image and each color image of the sample image, respectively. That is, the RGB values of all the pixels of the reference image correspond to P _L , the luminance of a certain color system image (red image, green image, blue image) is P _H , and the luminance of other channels is There corresponding to _{P L.}

Therefore, in the input luminance level of P _L and P _H is narrow, because it affects the accuracy of the gradation overall brightness correction, it is necessary to determine the correction parameters can widest possible linear approximation. The second embodiment aims to solve this problem.

  FIG. 6 is a block diagram showing a configuration of the projection luminance adjusting apparatus 1 according to the second embodiment of the present invention. It should be noted that portions corresponding to those in FIG. Below, the model evaluation part 8 different with respect to the structure of 1st Embodiment is demonstrated.

(Model evaluation unit 8)
In the first embodiment, the RGB channel values of the gray image are all set to (P _L , P _L , P _L ), and the RGB channel values of the red image are set to (P _H , P _L , P _L ). Assume that the RGB channel values of the green image are set to (P _L , P _H , P _L ), and the RGB channel values of the blue image are set to (P _L , P _L , P _H ).

Figure 7 is a conceptual diagram for explaining a method of setting the input brightness level P _L and P _H in the second embodiment. As shown by "●" in FIG. 7, in the second embodiment, in the initialization, to start the process from as wide a range as possible, to a value close to P _L is 0 level, P _H is a value close to 255 levels Is set. In this initial stage, since there is no correction parameter, the processing is completed only by preparing these sample images (resetting the sample images).

A correction parameter is calculated from this initial sample image by the same method as in the first embodiment. When the correction parameter of the brightness correction DB 7 is updated, the model evaluation unit 8 is restarted. The model evaluation unit 8 first prepares N test images for calculating an evaluation error. The RGB channel values of each test image are set to (P _L + n × ΔL, P _L + n × ΔL, P _L + n × ΔL), respectively. n is an integer of 0, 1, 2,..., N−1, and ΔL is a gradation quantization step determined by the following equation (17), rounded to the extent that the maximum gradation level is not exceeded. ing.

  FIG. 8 is a flowchart for explaining the operation of the model evaluation unit 8 according to the second embodiment. The model evaluation unit 8 determines whether or not there is a correction parameter in the luminance correction DB 7 (step S20). If there is a correction parameter (YES in step S20), the model evaluation unit 8 sets the luminance gradation according to the equation (17). Then, N types of test images are generated as the projection conversion images so as to be projected onto a predetermined region using the planar projection conversion data from the projection conversion DB 6 (step S21). Next, the model evaluation unit 8 corrects the luminance of all the test images using the correction parameter obtained from the luminance correction DB 7 (step S22).

  Next, the model evaluation unit 8 gives the corrected test images to the luminance update unit 4 and sequentially projects them through the projector 10 (step S23). The projected images are observed with the camera 20 one after another, and the average value (luminance evaluation value) of the luminance error with the original test image is calculated (step S24). Next, it is determined whether or not the luminance evaluation value is smaller than a predetermined error δ (for example, δ = 10 is set) (step S25).

  If the luminance evaluation value is smaller than the predetermined error δ (YES in step S25), the process of the model evaluation unit 8 is stopped and the process is terminated. On the other hand, if there is no correction parameter (NO in step S20), or if the luminance evaluation value is greater than or equal to a predetermined error δ, the sample image is reset (step S26). In this process, the RGB channel values of the previously set gray-based image, red-based image, green-based image, and blue-based image are changed. As indicated by “◯” in FIG. 7, the gradation is narrowed, that is, the following equations (18) and (19) are changed.

  ΔD is an arbitrary positive integer, for example, ΔD = 10. Along with this change, the pixels of the sample image are updated, and similarly, correction parameters are calculated from these sample images again according to the first embodiment. When the correction parameter is calculated again, it is possible to search for a linear approximation closest to the tone curve by repeating the process of the model evaluation unit 8, and to obtain a correction parameter corresponding to the linear approximation.

  According to the first and second embodiments described above, the output luminance P from the projector and the observation luminance by the camera can be obtained even if an arbitrary sample image is given without considering the output performance, camera sensitivity, dynamic range, etc. of the projector. Since it is possible to automatically obtain a color mixing matrix for linear approximation with C, it is possible to flexibly correct the brightness of the projector / camera system.

  In the present invention, a storage medium storing software program codes for realizing the functions of the first and second embodiments described above is supplied to a system or apparatus, and the CPU (MPU) of the system or apparatus is provided. It can also be realized by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and a storage medium storing the program code, for example, a CD-ROM, DVD-ROM, CD-R , CD-RW, MO, HDD, etc. constitute the present invention.

DESCRIPTION OF SYMBOLS 1 Projection brightness adjustment apparatus 2 Image input part 3 Brightness change detection part 4 Startup update part 5 Correction coefficient calculation part 6 Projection transformation DB
7 Brightness correction DB
8 Model Evaluation Unit 10 Projector 20 Camera

Claims (7)

A projection brightness adjustment method in a projector / camera system that captures an image projected by a projector with a camera,
An observation image acquisition step of acquiring a projection image obtained by projecting a sample image based on a set value by the projector as an observation image observed by the camera;
A luminance change detecting step for detecting a luminance change between the sample image and the observed image;
A brightness control step of controlling the brightness of the projector based on the detected amount of brightness change without using a correction parameter set in advance based on the performance of the projector or the camera. Projection brightness adjustment method. Repeatedly executing the observation image acquisition step and the luminance change detection step;
The control step includes
If the amount of change in luminance is equal to or greater than a predetermined allowable value, after updating the luminance input to the projector, projecting an image again based on the updated luminance from the projector;
The completion determination step of determining that the control of the luminance of the projector with respect to the sample image is completed when the amount of the luminance change is smaller than the predetermined allowable value. Projection brightness adjustment method.   The projection luminance adjustment method according to claim 1, further comprising a calculation step of calculating a correction parameter including a color mixing matrix and an ambient light vector after the luminance control step. Generating a test image for calculating a gradation error, and correcting the test image using the correction parameter;
Calculating a gradation error representing a luminance error in each gradation between a test observation image obtained by capturing the projected image of the test image after correction with the camera and the test image before correction;
If the gradation error is greater than or equal to a predetermined threshold, changing the luminance gradation of the sample image used for calculating the correction parameter, and correcting the sample image again by the luminance control step;
Determining that the correction of the sample image is completed when the gradation error is smaller than a predetermined threshold;
The projection luminance adjustment method according to claim 3, further comprising: A projection brightness adjusting device for a projector / camera system that captures an image projected by a projector with a camera,
An observation image acquisition unit for acquiring a projection image obtained by projecting a sample image based on a set value by the projector as an observation image observed by the camera;
A luminance change detection unit for detecting a luminance change between the sample image and the observation image;
A brightness control unit that controls the brightness of the projector based on the detected amount of brightness change without using a correction parameter that is set in advance based on the performance of the projector or the camera. Projection brightness adjustment device.   A computer program for causing a computer to execute the projection luminance adjusting method according to claim 1.   A recording medium on which a computer program for causing a computer to execute the projection luminance adjusting method according to claim 1 is recorded.

Images