JP2007049269A - Display image photography method and instrument - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct a variation in luminance caused by the flicker of a projector, and a variation in luminance caused by the fluctuation of the projector. <P>SOLUTION: This display image photography instrument detects a flicker period of a projector, controls a shutter time of a camera 6 for calibration to integer times of a detected flicker period, detects a fluctuation period corresponding to a variation in luminance caused by the illumination light of the projector or the heat of a cabinet, determines the number of times of iterative photography corresponding to the fluctuation period based on a shutter speed, repeats the photography of an image projected from the calibration camera 6 to a screen 1 by the determined number of iterative times using the shutter time equal to integer times of the flicker period, and creates an image subjected to flicker correction and fluctuation correction by averaging the images picked up by the number of iterative photography times. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a display image photographing method for photographing an image projected and displayed on a display surface by a projecting means, and a display image photographing apparatus for photographing an image projected and displayed on a display surface by a projecting means. .

Conventionally, there has been proposed a multi-display device that displays an entire image by projecting divided images from a plurality of projectors onto a screen. In this multi-display apparatus, an overlap portion is provided at the end of each divided image, and an adjacent overlap portion is bonded to form one whole image. Is possible. This multi-display device is configured by using a plurality of projectors having the same specification, but in addition to geometrical deviation caused by positional accuracy when the projectors are arranged, variations in projector components and brightness of the illumination lamp Due to variations in color, color variations, brightness (luminance) variations, white balance differences, and the like may occur. In that case, when displaying the whole image by overlapping multiple divided images and pasting them together, the connected part becomes conspicuous due to the variation in color and brightness between the divided images, and the display quality is impaired. End up. Therefore, in such a conventional multi-display device, a calibration camera (for example, a digital camera) is installed in front of the screen, an image (for example, a test pattern image) displayed on the screen is captured, and the captured image data is converted into captured image data. Based on this, correction data (for example, a profile indicating the color characteristics of each projector) is generated, and when a partial image is projected from each projector, color correction and geometric correction are performed with reference to each profile. .
As another conventional technique for creating a profile, there is a system in which a test pattern image is projected on a screen and photographed by photographing means such as a digital camera, and a profile is created from photographing data. In the following, taking a test pattern image with a digital camera and creating a profile is referred to as “calibration”, the camera used for this purpose is referred to as “calibration camera”, and the system used for this is referred to as “calibration”. Will be referred to as a “system”.

  When acquiring color correction data in the calibration system applied to the multi-display device, a test pattern is sent from each PC to each projector, and tests with different brightness and color are performed on the screen from each projector. When the pattern is projected in time series, the test pattern displayed on the screen is photographed by the calibration camera, the photographed data is stored in the PC, and the correction data is calculated in the PC. The correction data obtained in this way is sent to the image correction processing unit, where the image data used for actual display is corrected in real time and sent to each projector. Therefore, by projecting the image incorporating the correction, Then, an image whose color and brightness are corrected is displayed on the screen. The color correction includes gamma correction for each small area or pixel for reducing color unevenness between projectors (or color unevenness within each projector), R (Red), G (Green), and B ( Blue) There is white balance correction for matching the display level of each color.

When acquiring the gamma correction data in the calibration system, after setting the shutter speed in the calibration camera (for example, a digital camera) so that the luminance level is in an appropriate range, the minimum level (black) in each RGB color After projecting a plurality of images (test patterns) in increments of a predetermined luminance value from the first to the highest level onto the screen, each image is photographed, and after detecting the luminance level on the PC, the luminance level difference of each projector Is calculated, and correction data that eliminates the brightness level difference between the projectors is created.
On the other hand, when acquiring white balance correction data, a shutter speed that sets the luminance level within an appropriate range is set in a calibration camera (for example, a digital camera), and then, for example, CIE 1931 is used on the calibration camera side. XYZ color system A filter that passes through the X, Y, and Z wavelength regions of the color matching function, with an X filter, a Y filter, and a Z filter sequentially inserted in the camera optical path, an image (test) As a pattern), a test pattern of R brightness value 255 (all red), a test pattern of G brightness value 255 (all green), and a test pattern of B brightness value 255 (all blue) are sequentially projected on the screen, Each image is taken, and correction data is created so that each color is balanced on the PC.

When shooting a plurality of images (test patterns) in order to obtain gamma correction data and white balance correction data in the calibration system described above, exposure such as shutter speed adjustment is performed in a calibration camera (for example, a digital camera). Although it is necessary to perform control, the projection screen of the projector has the following three specific luminance changes.
(1) Flicker by display cycle Since the single-plate DLP projector sequentially projects RGB in time series, the luminance changes at the timing of each color, for example, 1/60 second or 1/120 second per color. It is lit at the cycle.
(2) Luminance change due to fluctuations caused by the light source of the projector and the heat of the housing Discharge tube type light source using discharge between electrodes such as an ultra-high pressure mercury lamp and a xenon lamp generally used as a light source for projector illumination Since fluctuations in the radiation intensity occur due to the influence of the convection of the enclosed gas, the luminance distribution of the illumination light projected from such a light source becomes unstable. Further, the brightness of the projected image changes in an unstable manner due to the fluctuation of the air on the optical path caused by the heat generated from the projector housing.
(3) Light source flicker due to sinusoidal high-frequency power A sinusoidal high-frequency power obtained by increasing AC power (50 Hz) several times faster is used as the projector power. It is a brightness change. Since the period of (3) is a period relatively faster than the periods of (1) and (2), the influence of (3) is an average if the low shutter speed corresponding to the above (1) and (2) is used. It will be almost disappeared.
As described above, if shooting is not performed under appropriate exposure conditions that take into account flicker that causes luminance changes due to electrical processing and fluctuations that cause luminance changes due to heat of the light source and the housing, the desired average An image deviating from “brightness” is taken. More specifically, when shooting while gradually changing the brightness from black to white, accurate brightness change data cannot be obtained if it is affected by flicker or fluctuations. This causes a problem such as the brightness level not matching.

  As a conventional technique for countermeasures against flicker, the frequency analysis unit performs frequency analysis on the detection output of the illumination light detection unit that detects a change in the illumination light of the subject, and an integer of the cycle of the most frequent frequency component in the output of the frequency analysis unit The exposure time is calculated as an exposure time, and the exposure time is given to an imaging device that captures an optical image of the subject (see, for example, Patent Document 1, hereinafter referred to as Prior Art 1), the first image signal, and the first Amplifying means for amplifying the image signal and outputting the second image signal; a first signal level obtained by sampling the first image signal in the first period; and a second signal shorter than the first period. Gain control means for controlling the gain of the amplifying means based on a second signal level sampled at a period of 2, wherein the second period was used for taking an image of the first image signal Light Image signal fluctuation compensating circuit corresponding to the period of fluctuation of the radiation intensity of use the light source (for example, see Patent Document 2, hereinafter referred to as prior art 2) is. In addition to the above, as a basic flicker countermeasure, there is “matching the shutter speed to the flicker cycle”.

JP-A-7-336586 JP 2004-141382 A

Since the prior art 1 is a measure assuming that the cause of flicker is a relatively high frequency and a constant frequency such as a fluorescent lamp (100 Hz), it cannot cope with the case where the fluctuation cycle is slower (longer) than the shutter time. There is a problem.
In the prior art 2, the gain of the amplifying means is controlled (increased) instead of changing the shutter time in accordance with the flicker cycle, so that the S / N ratio of the image signal deteriorates as the gain increases. There is a problem that.
Furthermore, since both of the above prior art 1 and prior art 2 are techniques for taking countermeasures only for flicker, there is a problem in that they do not cope with “brightness change due to fluctuations caused by the light source of the projector or the heat of the housing”. is there.

A first object of the present invention is to provide a display image photographing method capable of correcting both a luminance change caused by flicker of a projection unit and a luminance change caused by fluctuation of the projection unit.
A second object of the present invention is to provide a display image photographing apparatus capable of correcting both a luminance change caused by flicker of a projection unit and a luminance change caused by fluctuation of the projection unit.

  In order to achieve the first object, a first invention according to claim 1 is a display image photographing method for photographing an image projected and displayed on a display surface by a projecting unit, using the photographing unit, the projecting unit. A flicker cycle detecting step for detecting a flicker cycle of the image capturing device, a shutter time control step for controlling a shutter time of the photographing unit to an integral multiple of the flicker cycle, a luminance change caused by illumination light of the projection unit, or the projection unit A fluctuation cycle detection step for detecting a fluctuation cycle that is a luminance change caused by heat of the housing, etc., a shooting number determination step for determining the number of repeated shootings corresponding to the fluctuation cycle based on the shutter time, and the flicker cycle An image projected and displayed on the display surface by the photographing means using a shutter time that is an integral multiple of And repeating imaging step for only shooting times repeatedly photographed, and performs the photographing image averaging process, sequentially averaging the repeatedly shooting number of times shot image obtained by the repetitive imaging process.

  According to a second aspect of the present invention, the flicker cycle detection step includes a first step of photographing the image projected and displayed on the display surface by the photographing means a plurality of times using the same high shutter speed. A second step of obtaining a level difference by comparing signal levels of a plurality of captured images taken at the same shutter speed by the photographing means; and the photographing means using the same shutter speed that is lower than the same shutter speed. A third step of photographing the image projected and displayed on the display surface a plurality of times is compared with a signal level of a plurality of photographed images photographed by the photographing means at the same low shutter speed to obtain a level difference. Comparison between the fifth step and the fifth step for comparing the previous level difference in the fourth step with the current level difference in the fourth step While the previous level difference is greater than or equal to the current level difference, the shutter speed is gradually shifted to the lower speed side and the third step to the fifth step are repeated, and the comparison between the sixth step and the fifth step is the previous step. And a seventh step of detecting a shutter speed corresponding to the previous level difference as a flicker cycle of the projection means when the level difference is less than the current level difference.

  According to a third aspect of the present invention, in the shutter time control step, the incident light amount control unit that restricts the incident light amount to the photographing unit so that the signal level of the photographed image does not saturate is limited. Control is performed by a combination of the state and the shutter time.

  According to a fourth aspect of the present invention, the fluctuation cycle detecting step is projected and displayed on the display surface by the photographing unit using a shutter time controlled to be an integral multiple of a flicker cycle by the shutter time control step. A photographing process for performing the process X and the process Y, which is a process of photographing the captured image N (N = 1, 2,...) Times, a photographed image obtained by the process X, and a photographed image obtained by the process Y, respectively. An averaging process for averaging, a level difference calculating process for obtaining a level difference by comparing the levels of the averaged captured image of the process X and the averaged captured image of the process Y, A level difference comparison step for comparing the level difference in the case of (N-1) and the level difference in the case where the number of shootings is N, and the level difference in the case where the number of shootings is N in the level difference comparison step. When the number of shadows is smaller than the level difference in the case of (N−1), the number of times of photographing in the level difference comparing step and the re-photographing step of repeating the photographing step to the level difference comparing step with the number of photographing times as (N + 1) When the difference in level when N is larger than the difference in level when the number of times of shooting is (N-1), the number of times of shooting is determined to be (N-1), and the number of times of shooting is determined. The product of the shutter speed in the photographing process and the number of photographing times determined in the photographing number determining step is detected as the fluctuation cycle.

  In a fifth aspect of the present invention, the fluctuation cycle detection step is performed a plurality of times to detect the fluctuation cycle a plurality of times, and is defined by [(fluctuation cycle maximum value−fluctuation cycle minimum value) / fluctuation cycle maximum value]. The repetitive photographing step is performed for the number of repetitive photographing determined based on the reciprocal of the fluctuation deviation to be performed.

  According to a sixth aspect of the present invention, an elapsed time-fluctuation period table in which characteristics of a fluctuation period with respect to an elapsed time of the illumination light source of the projection unit are stored in advance, and characteristics of fluctuation periods with respect to the temperature of the projection unit are stored in advance. A fluctuation cycle is created by creating at least one of the temperature-fluctuation cycle table and referring to the elapsed time-fluctuation cycle table and the temperature-fluctuation cycle table using at least one of the elapsed time and the temperature as a parameter at the time of shooting. It is characterized by calculating | requiring.

  According to a seventh aspect of the present invention, there are variations in captured images when the image projected and displayed on the display surface is captured a plurality of times by the display image capturing method of the sixth aspect, and the first to fourth aspects of the present invention. A display image photographing method having a smaller variation is selected by comparing the variation in the photographed image when the image projected and displayed on the display surface is photographed a plurality of times by any one of the display image photographing methods. And

  In order to achieve the first object, an eighth invention according to claim 8 is a display image photographing method for photographing an image projected and displayed on a display surface by a projecting means, using the photographing means, wherein the projecting means A flicker cycle detecting step for detecting a flicker cycle of the image capturing device, a shutter time control step for controlling a shutter time of the photographing unit to an integral multiple of the flicker cycle, and a luminance change caused by illumination light of the projection unit by a fluctuation cycle detecting unit. Alternatively, a fluctuation period detecting step of detecting a fluctuation period that is a luminance change caused by heat of the casing of the projection means and a shutter time that is an integral multiple of the flicker period are projected onto the display surface by the photographing means. A repetitive photographing step of repeatedly photographing the displayed image during the fluctuation period, and before being obtained by the repetitive photographing step. A captured image averaging step of averaging the captured image of the number corresponding to the fluctuation cycle, characterized in that sequentially performed.

  In order to achieve the second object, a ninth invention according to claim 9 is a display image photographing apparatus for photographing an image projected and displayed on a display surface by a projection means, and projects the image onto the display surface. An imaging unit that captures the displayed image and outputs a captured image, a flicker cycle detection unit that detects a flicker cycle of the projection unit, and a shutter time that controls the shutter time of the imaging unit to an integral multiple of the flicker cycle Based on the shutter time based on the shutter time, the control means, the fluctuation period detecting means for detecting the fluctuation period that is the luminance change caused by the illumination light of the projection means or the brightness of the casing of the projection means, etc. The number-of-times-determining means for determining the number of times of repeated shooting corresponding to the fluctuation cycle, and the shooting means using a shutter time that is an integral multiple of the flicker cycle. Repeated photographing means for repeatedly photographing the image projected and displayed on the display surface for the determined number of repeated photographing, and photographed image averaging for averaging the photographed images for the number of repeated photographing obtained by the repeated photographing means And means.

  According to a tenth aspect of the present invention, a sensor functioning as the flicker period detecting means and the fluctuation period detecting means is provided separately.

  According to the first aspect of the invention, when the image projected on the display surface by the projection unit is photographed by the photographing unit, the flicker cycle of the projection unit is detected, and the shutter time of the photographing unit is set to the flicker cycle. Controlled by an integral multiple, detects a fluctuation cycle that is a luminance change caused by illumination light of the projection unit or a luminance change caused by heat of the housing of the projection unit, and the fluctuation period is determined based on the shutter time. After determining the corresponding number of repeated shooting, the image projected and displayed on the display surface by the shooting unit using a shutter time that is an integral multiple of the flicker cycle is repeatedly shot for the determined number of repeated shooting, Since the number of the captured images for the number of repeated shooting obtained by the repeated shooting process is averaged, it is caused by flicker of the projection means. Display image capturing method capable of correcting both luminance change due to fluctuations in degrees changes and projection means can be provided.

  According to the second invention, in the flicker cycle detecting step, the image projected and displayed on the display surface by the photographing unit is photographed a plurality of times using the same high shutter speed, and the photographing unit performs the same shutter speed. The signal levels of a plurality of captured images are compared to determine a level difference, and the plurality of images projected and displayed on the display surface by the imaging unit using the same shutter speed that is lower than the same shutter speed. The level difference is obtained by comparing the signal level of a plurality of shot images taken at the same slow shutter speed by the photographing means, and the previous level difference in the second step and the current step in the fourth step. The shutter speed is gradually decreased while the previous level difference is greater than or equal to the current level difference in the comparison of the fifth step. When the previous level difference is less than the current level difference in the comparison of the fifth step, the shutter speed corresponding to the previous level difference is set to the shutter speed. Since it is detected as the flicker cycle of the projection means, the shutter speed corresponding to the flicker cycle of the projection means can be obtained.

  According to the third invention, the shutter time control step includes an incident light amount limiting state of the incident light amount control means for limiting the incident light amount to the photographing means so that the signal level of the photographed image is not saturated, and the shutter Since control is performed in combination with time, the signal level (brightness level) of the photographed image taken by the photographing means falls within an appropriate range so as not to be saturated. Fluctuation correction can be performed.

  According to the fourth invention, in the fluctuation cycle detecting step, the image projected and displayed on the display surface by the photographing means using the shutter time controlled to an integral multiple of the flicker cycle by the shutter time control step is N. Steps X and Y, which are steps (N = 1, 2,...), Are performed, the captured image obtained by the step X and the captured image obtained by the step Y are respectively added and averaged, and the step X A level difference is obtained by comparing the levels of the averaged captured image and the averaged captured image of the process Y, and the level difference and the number of times of shooting when the number of times of shooting is (N-1) are N. If the number of shootings is N in the level difference comparison step is smaller than the level difference when the number of shootings is (N-1), the number of shootings is set to (N + 1). Then, the imaging step to the level difference comparison step are repeated, and when the level difference in the level difference comparison step is larger than the level difference in the case where the number of times of photographing is N, the number of times of photographing is (N−1). The number of times of photographing is determined as (N-1), and the product of the shutter speed in the photographing step and the number of times of photographing determined in the number of times of photographing determination is detected as the fluctuation period. The fluctuation cycle of fluctuation caused by the light source or the heat of the casing can be obtained.

  According to the fifth invention, the fluctuation cycle detection step is performed a plurality of times to detect the fluctuation cycle a plurality of times, and the fluctuation deviation defined by [(fluctuation cycle maximum value−fluctuation cycle minimum value) / fluctuation cycle maximum value] The number of times of repeated shooting determined based on the reciprocal of the above is performed, so that the fluctuation cycle detection value can be solved and the fluctuation cycle can be accurately obtained, and the brightness caused by the fluctuation of the projection means. Changes can be accurately corrected.

  According to the sixth aspect of the present invention, the elapsed time-fluctuation period table in which the characteristics of the fluctuation period with respect to the elapsed time of the illumination light source of the projection unit are stored in advance, and the temperature-fluctuation in which the characteristics of the fluctuation period with respect to the temperature of the projection unit are stored in advance. Since at least one of the period tables is created and the fluctuation period is obtained by referring to the elapsed time-fluctuation period table and the temperature-fluctuation period table using at least one of the elapsed time and the temperature as a parameter at the time of shooting, By replacing the fluctuation cycle detection step based on the actually measured data in the first invention with a fluctuation cycle detection step based on the reference of the elapsed time-fluctuation cycle table and temperature-fluctuation table based on the above parameters, it is possible to expect high-speed processing. Become.

  According to the seventh aspect of the present invention, the variation in the captured image when the image projected and displayed on the display surface is captured a plurality of times by the display image capturing method of the sixth aspect, and any one of the first to fourth aspects Compared with the variation of the captured image when the image projected and displayed on the display surface by the display image capturing method is captured a plurality of times, the display image capturing method with the least variation is selected, so that the flicker of the projection means Both the resulting luminance change and the luminance change caused by the fluctuation of the projection means can be corrected reliably.

  According to the eighth aspect of the invention, when an image projected and displayed on the display surface by the projection unit is captured by the imaging unit, the flicker cycle of the projection unit is detected, and the shutter time of the imaging unit is set to the flicker cycle. An integer multiple of the flicker period is detected by detecting a fluctuation period which is a luminance change caused by illumination light of the projection means or a brightness change caused by heat of the casing of the projection means, etc. An image projected and displayed on the display surface by the photographing unit using a double shutter time is repeatedly photographed during the fluctuation cycle, and the number of photographed images corresponding to the fluctuation cycle obtained by the repeated photographing step is averaged. Therefore, a table that can correct both the luminance change caused by the flicker of the projection unit and the luminance change caused by the fluctuation of the projection unit. It is possible to provide an image capturing method.

  According to the ninth invention, the display image photographing apparatus for photographing the image projected and displayed on the display surface by the projecting means, and the photographing means for photographing the image projected and displayed on the display surface and outputting the photographed image. Flicker cycle detection means for detecting the flicker cycle of the projection means, shutter time control means for controlling the shutter time of the photographing means to be an integral multiple of the flicker cycle, and luminance change caused by illumination light of the projection means or Fluctuation period detecting means for detecting a fluctuation period that is a luminance change caused by heat or the like of the housing of the projection means, and a photographing number determining means for determining the number of repeated photographing corresponding to the fluctuation period based on the shutter time. The image projected and displayed on the display surface by the photographing unit using a shutter time that is an integral multiple of the flicker cycle is determined. A flicker of a projection means, comprising: a repetitive photographing means for repetitively photographing the number of repetitive photographing times; and a photographed image averaging means for averaging the photographed images for the number of repetitive photographing obtained by the repetitive photographing means. Therefore, it is possible to provide a display image photographing apparatus capable of correcting both the luminance change caused by the luminance change and the luminance change caused by the fluctuation of the projection means.

  According to the tenth invention, the flicker cycle detecting means and the sensor functioning as the fluctuation cycle detecting means are separately provided. Therefore, in the fifth invention, it is desired to speed up the processing for generating the picked-up image subjected to the flicker correction and the fluctuation correction. In this case, the shutter speed, the exposure condition, and the number of times of photographing of the photographing unit are obtained by using the luminance change caused by the flicker cycle of the light source of the projection unit detected by the sensor and the luminance change caused by the fluctuation of the projection unit. Therefore, the time required for flicker correction and fluctuation correction can be shortened, and the processing for generating a captured image that has been subjected to flicker correction and fluctuation correction can be speeded up.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing an overall configuration of a display image photographing apparatus used for carrying out a display image photographing method according to a first embodiment of the present invention, and FIG. 2 is a calibration camera used in the display image photographing apparatus according to the first embodiment. It is a figure which shows the detailed structure of a personal computer.

  The display image capturing device of the present embodiment captures the image when performing color correction (gamma correction and white balance correction) in order to adjust the display state of the image projected on the screen of the multi-display device, for example. The flicker correction and the fluctuation correction are performed based on the photographed image. As shown in FIG. 1, a plurality of units (in the illustrated example) project and display divided images so as to have an overlap portion 2 on the screen 1. The projectors 3a, 3b, 3c, and 3d, which are four projectors), the image correction processing unit 4 that supplies the divided image signals to the projectors 3a, 3b, 3c, and 3d, and the image correction processing unit 4, respectively. A plurality of images (hereinafter also referred to as test patterns) for correcting variations in the image display state among the projectors 3a, 3b, 3c, 3d. ) Are sequentially projected and displayed on the screen 1, and a flicker correction function that generates captured image data that has been subjected to flicker correction and fluctuation correction by continuously shooting an image displayed on the screen 1 a plurality of times as will be described later. And a personal computer (PC) 5 having a fluctuation correction function, a calibration camera 6 which is a photographing means for photographing the plurality of images under a predetermined exposure condition defined by a shutter speed and an aperture, and a calibration camera 6 and a color filter mechanism 8 provided at a predetermined position on the same optical path.

  The image correction processing unit 4 includes a correction data storage unit 4a for storing correction data created by the PC 5, and divided image data (which may be supplied by the image correction processing unit 4 itself). And a correction processing unit 4b that outputs the corrected data by adding the correction by the correction data.

  As shown in FIG. 2, the PC 5 detects a luminance level (signal level) of “photographed image data of an image projected and displayed on the screen 1” input from the calibration camera 6. Then, based on the “photographed image data of the image projected and displayed on the screen 1”, correction data relating to the projector that projected and displayed the image is calculated (generated) and stored, and the “projected and displayed on the screen 1” A correction data calculation storage unit 5b for calculating (generating) and storing correction data (flicker correction and fluctuation-corrected captured image data) on the basis of “photographed image data obtained by continuously shooting an image a plurality of times”, and calibration Obtained when shooting continuously with the same shutter speed by the camera 6 A luminance level comparison unit 5c that compares the luminance levels of the captured images to calculate a luminance level difference, and exposure control (shutter speed) of the calibration camera 6 so that the luminance level difference calculated by the luminance level comparison unit 5c is minimized. Command, aperture command), and the calibration camera 6 sets the number of times of repeated shooting to be described later based on the shutter speed at which the brightness level difference calculated by the brightness level comparison means 5c is minimized. (Command) the number-of-shooting setting unit 5e, and a color filter switching control unit 5f that selectively switches color filters used when sequentially shooting the plurality of images based on the luminance level detected by the luminance level detection unit 5a. And comprising. Note that the color filter control unit 5f performs CIE 1931 XYZ color system color matching function X during white balance correction, as will be described later, during a series of imaging periods of the plurality of test pattern images by the calibration camera 6. , Y, and Z, which are filters that pass through the wavelength regions of Y, Z, and Z, are sequentially switched and used, and filter switching control that uses a cavity without a color filter is performed during gamma correction.

  As the calibration camera 6, a monochrome digital camera which is a monochrome photographing unit is used in the present embodiment. As shown in FIG. 2, the calibration camera 6 includes a lens mechanism 6a that forms an image of a subject, a diaphragm mechanism 6b that adjusts the amount of light emitted from the lens mechanism 6a, and a subject that is incident through the diaphragm mechanism 6b. A CCD 6c that photoelectrically converts the image of the image into an electrical signal (image data signal), an S / H • AGC circuit 6d that performs sample hold and automatic gain control (AGC) on the image data signal output from the CCD 6c, An A / D converter 6e for A / D converting the image data signal output from the H • AGC circuit 6d, a camera signal processor 6f for performing various signal processing on the A / D converted image data signal, and a camera An input / output I / F unit 6g that outputs an image data signal output from the signal processing unit 6f to the PC 5 and relays various signals input from the PC 5, A CCD drive unit 6h that drives the CCD 6c at the shutter speed commanded from the light control unit 5d, and an aperture drive unit 6i that drives the aperture mechanism 6b so that the aperture value commanded from the exposure control unit 5d of the PC 5 is obtained. It comprises. In the present embodiment, a camera that can set the shutter speed to, for example, 1/1000 second 8 to 2 seconds is used as the calibration camera 6. As the calibration camera 6, in addition to a monochrome digital camera, a monochrome video camera, a monochrome CMOS sensor, a monochrome line sensor, or the like, which is a monochrome photographing unit, can also be used.

The color filter mechanism 8 is used when photographing a test pattern for white balance correction, and is an X filter 8a that is a filter that passes through the X, Y, and Z wavelength regions of the CIE 1931 XYZ color system color matching function. , Y filter 8b, Z filter 8c, a turret 8d having four openings for mounting these filters on the circumference, and a filter (or cavity) corresponding to a command from the color filter switching control unit 5f of the PC 5 ) Is a motor (not shown) that drives the turret 8c to rotate around the central axis so that it is located on the optical path. Used when shooting images for gamma correction that are not used.
In the color filter mechanism 8, the CIE 1931 XYZ color system color matching function is used as a plurality of color filter means capable of separating the input light of the calibration camera 6 into a plurality of wavelength characteristics for white balance correction. X filters, Y filters, and Z filters, which are filters that pass through each wavelength region of X, Y, and Z, are used, but any color filter can be used even if R, G, and B pass filters are used instead. It may be used.

Next, flicker correction and fluctuation correction performed at the time of calibration in the display image photographing apparatus of the present embodiment will be described. In the display image photographing apparatus of the present embodiment, color correction and geometric correction are performed as calibration. The procedure is as follows.
First, as shown in FIG. 1, the calibration camera 6 is arranged at the observation position of the general audience. Next, a test pattern image for geometric correction is projected and displayed on the screen 1 and photographed, and geometric correction data is created by the PC 5 based on the photographed data (in the case of a multi-display device in which geometric deviation does not occur, geometric correction is performed). Is not required).
Thereafter, a test pattern image for color correction (gamma correction and white balance correction) with geometric correction is projected and displayed on the screen 1, and color correction data is created by the PC 5 based on the shooting data. At that time, “if the flicker cycle of the light sources of the projectors 3 a, 3 b, 3 c, 3 d as the projecting means is slower (longer) than the shutter time of the calibration camera 6, “The brightness level of the captured image data varies” and “the fluctuation cycle of the air fluctuation on the optical path due to the heat of the housings of the projectors 3 a, 3 b, 3 c, 3 d as the projecting means is the calibration camera 6. If the shutter time is later (longer) than the shutter time, the luminance level of the captured image data photographed by the calibration camera 6 varies depending on the photographing timing. Correction and fluctuation correction ".

  In the color correction performed as described above, small area or pixel-by-pixel gamma correction for reducing color unevenness between projectors (or color unevenness in each projector), R (Red), G White balance correction is performed to match the display levels of the (Green) and B (Blue) colors. The small area to be corrected in gamma correction can be determined according to the number of pixels of the calibration camera (digital camera) to be used. For example, the number of display pixels on the screen 1 is 8 million. When the number of pixels of the digital camera is 2 million pixels, every 2 × 2 pixels is a correction area. In the gamma correction, a plurality of test pattern images in predetermined luminance value increments in the range from the lowest level (black) to the highest level of the luminance signal are projected and displayed on the screen 1 for each RGB color, and are displayed on the calibration camera 6. Shoot with. On the other hand, the white balance correction is performed by projecting and displaying a test pattern image having appropriate luminance values (for example, maximum luminance values) for each color of RGB on the screen 1 to match the human visual sensitivity such as the CIE 1931 XYZ color system color matching function. A test pattern image is photographed by the calibration camera 6 through the filter, and correction is performed so as to balance each color based on the photographing data.

Next, flicker correction and fluctuation correction performed in the display image photographing apparatus of the present embodiment will be described in detail with reference to FIGS.
FIG. 3 is a flowchart showing an outline of processing for flicker correction and fluctuation correction performed in the display image photographing apparatus of the first embodiment. It should be noted that the processing for flicker correction and fluctuation correction in FIG. 3 is performed at the same time when the test pattern image for color correction is captured, but is performed prior to the test pattern image for color correction. You may make it do.

  First, in step S1 of FIG. 3, a shift between a flicker period of a light source of a projector as a projection unit (that is, a vertical display period of the projector and a shutter time of the calibration camera 6) is performed by a process shown in a flowchart of FIG. In the next step S2, the shutter time of the calibration camera 6 as the photographing means is controlled (determined) to an integral multiple of the flicker cycle obtained in the above step S1. Set the shutter time corresponding to the flicker cycle. In this embodiment, the detection method shown in the flowchart of FIG. 4 to be described later is used as the flicker cycle detection method. Instead, the detection method described in Patent Document 1 and Patent Document 2 described above are used. A known technique such as a detection method may also be used.

  In the next step S3, exposure adjustment is performed. This exposure adjustment limits the amount of light incident on the calibration camera 6 so that the signal level of the captured image does not become saturated (for example, 70% to 90%). And the shutter time determined in step S2 above. In the present embodiment, the exposure adjustment is performed in step S3. However, if the signal level is saturated or the signal level is low and a problem occurs in step S1 or step S2, the process in step S1 is performed. Temporary exposure adjustment may be performed before the signal level becomes appropriate.

  In the next step S4, both the fluctuation cycle (brightness change caused by the light source of the projector or the brightness change due to the fluctuation of the air on the optical path caused by the heat of the projector housing, etc.) are performed by the processing shown in the flowchart of FIG. And the number of times of repeated shooting corresponding to the fluctuation cycle is determined. For example, when the detected fluctuation cycle is 1 second and the shutter time determined in step S2 is 1/60 seconds, the number of repeated shooting is 1 second / (1/60). Seconds) = 60 (times).

In the next step S5, actual photographing is performed. In this actual photographing, the image projected and displayed on the screen 1 by the calibration camera 6 is repeatedly photographed by the number of repeated photographing determined in step S4 using a shutter time that is an integral multiple of the flicker cycle determined in step S2. By doing. Then, in the next step S6, the photographic image data is averaged for the number of repeated photographing obtained by the repeated photographing in step S5. As a result, a captured image subjected to flicker correction and fluctuation correction is generated.
The photographing performed in steps S1 to S4 is referred to as “pre-photographing” because it is photographing for determining photographing conditions, and the photographing performed in step S5 is actually photographing for obtaining photographed image data. Therefore, we will call it “real shooting”.

Next, the detection of the flicker period in the present embodiment will be described based on the flowchart of FIG. In the flowchart of FIG. 4, since there is no comparison data at the first time, it is necessary to separately fetch comparison data for the first time. Note that the shutter speed of the calibration camera 6 can be set, for example, between 1/500 and 2 seconds.
[First comparison data import method]
Similarly to step S11 in FIG. 4 described later, the shutter speed is set to a predetermined shutter speed (for example, 1/500 second which is the upper limit high-speed shutter speed). However, if the signal level is almost 0 even when the aperture is opened because the image on the screen 1 as a subject is dark, the shutter speed is set to a shutter speed slower than 1/500 second. Next, similarly to step S12 of FIG. 4 described later, images projected and displayed on the screen 1 by the calibration camera 6 are continuously photographed at the shutter speed, and stored in the correction data calculation storage unit 5b of the PC 5. Two pieces of photographed image data are captured and stored in a frame memory (see FIG. 6 described later). Next, in the same manner as in step S13 of FIG. 4 described later, the two photographed image data are integrated to obtain respective signal levels (luminance levels), the level difference between the two signal levels is calculated, and the frame memory is calculated. To remember.

[Detection of flicker cycle after importing first comparison data]
First, in step S11 of FIG. 4, the shutter speed of the calibration camera 6 is set to a predetermined shutter speed. As this predetermined shutter speed, for example, 1/250 seconds, which is a shutter speed lowered by one step from the upper limit high-speed shutter speed (1/500 seconds) used in the first shooting described above, is used. In the next step S12, images projected and displayed on the screen 1 are continuously photographed at the shutter speed, and two pieces of photographed image data are stored in a frame memory (see FIG. 6 described later) in the correction data calculation storage unit 5b of the PC 5. Capture. In the next step S13, the two photographed image data are integrated to obtain respective signal levels (luminance levels), the level difference between the two signal levels is calculated, and stored in the frame memory.

  In the next step S14, the current level difference is compared with the previous level difference (at the time of shooting at shutter speed = 1/500 seconds), and if the current level difference is less than the previous level difference, processing is performed. The process proceeds to step S15, and if the current level difference is greater than or equal to the previous level, the process proceeds to step S16 to set the previous shutter speed as the flicker cycle. In step S15, it is determined whether or not the shutter speed can be decreased. If YES (can be decreased), the shutter speed is decreased by one step in the next step S17, and then the process returns to step S12 and subsequent steps. If NO (cannot be lowered), the process proceeds to step S16. The calibration camera 6 used in this embodiment can set a shutter speed in a range of 1/1000 second to 2 seconds, and the flicker cycle is generally substantially equal to the display cycle of the projector. Since it is almost 60 seconds or more, it is unlikely that step S16 will be NO, so if step S16 is NO, the user is warned as an “error” and the flicker cycle is detected again. You may make it prompt.

  Step S11 to Step S12 ... NO in Step S14-Step S16 completes detection of the flicker cycle. In Step S18 subsequent to Step S16, it is determined whether or not the detected flicker cycle is valid. In order to verify, it is determined whether or not the signal level of the two photographed image data is 100%. In this determination, “if the signal level of the two shot image data is 100%, the level difference is zero, but in this case, the two shot images are saturated and the level difference is zero”. Therefore, considering that the shutter speed detected in step S16 is inappropriate as the flicker cycle, the process proceeds to step S19 to perform exposure correction (exposure correction), and then the process is performed in step S12. After that, the flicker cycle is detected again. In the exposure correction in step S19, the combination of the aperture and shutter time of the calibration camera 6 is adjusted in order to limit the amount of incident light so that the signal levels of the two photographed image data are not saturated. To do. Specifically, if the signal level is saturated, the aperture value is increased to reduce the amount of incident light. If the signal level is small, the aperture value is decreased (the aperture is opened) and the shutter speed is approximated to the flicker cycle. Decrease step by step from the shutter speed (shutter speed obtained in step S16) 1/60 second → 1/30 second → 1/15 second → 1/8 second → 1/4 second → 1/2 second → 1 second Change it.

  The exposure adjustment in step S19 and the subsequent re-detection of the flicker cycle are repeated until the determination in step S18 becomes NO. When the determination in step S18 becomes NO, the process proceeds to step S20, and the shutter speed at that time To the shutter speed. The shutter speed determined in step S20 is a shutter speed that approximates the flicker cycle.

Next, detection of the fluctuation cycle in this embodiment will be described based on the flowchart of FIG. In the flowchart of FIG. 5, since there is no comparison data at the first time, it is necessary to separately acquire comparison data for the first time. In the example described below, it is assumed that the shutter time (that is, the shutter time for reducing flicker) of the calibration camera 6 determined in step S20 is 1/60 seconds and the fluctuation at that time. The period is assumed to be 1 second.
[First comparison data import method]
In step S21 of FIG. 5 to be described later, the number N of shootings (N is a natural number) is set to N = 1 only during the first pre-shooting. Next, similarly to steps S22 to S25 of FIG. 5 described later, the image projected and displayed on the screen 1 by the calibration camera 6 is continuously photographed N times with the shutter time determined in step S20. The two types of captured image data (captured image data X ₁ , captured image data Y ₁ ) are captured and stored in a frame memory (see FIG. 6 described later) in the correction data calculation storage unit 5 b of the PC 5. Next, in the same manner as in step S26 of FIG. 5 described later, the two types of captured image data are integrated to obtain respective signal levels (luminance levels), the level difference between the two signal levels is calculated, and the frame Store in memory.

[Detection of fluctuation cycle after first comparison data is imported]
First, in step S21 in FIG. Since the number of times of photographing N is set according to an increment of N (N = N + 1) in step S28 described later, the number of times of photographing N is 1 + 1 = 2 based on the previous N = 1 at the second pre-shooting. Times. Similarly, in the third pre-shooting, the number of shootings N is (2 + 1) = 3, and in the Nth pre-shooting, the number of shootings N is (N + 1). In the next step S22 and step S23, the image projected and displayed on the screen 1 by the calibration camera 6 is continuously photographed N times (in the second case, twice) with the shutter time determined in step S20. Then, two types of captured image data (captured image data X ₁ , X ₂ and captured image data Y ₁ , Y ₂ ) are taken into a frame memory (see FIG. 6 described later) in the correction data calculation storage unit 5 b of the PC 5. Remember. In the next step S24, the photographic image data X ₁ , X ₂ and the photographic image data Y ₁ , Y ₂ ) are respectively added and averaged. 5b is stored in the frame memory (see FIG. 6 described later). In the next step S26, the two types of photographed image data addition average values Xav and Yav are integrated to obtain each signal level (luminance level), and the level difference between the two signal levels is calculated and stored in the frame memory. To do.

  In the next step S27, the current level difference is compared with the previous level difference (the first time if this is the second time), and if the current level difference is less than the previous level difference, the process is stepped. The process proceeds to S28, and if the current level difference is greater than or equal to the previous level, the process proceeds to step S29. In step S28, after incrementing N (N = N + 1), the process returns to step S21 and the subsequent steps to increase the number of shootings by one, and the same processing as described above is performed. In step S29, the number of shootings is set to (N− 1) Decide at times. In this example, since the fluctuation cycle is 1 second, N = 60 or the number of shootings.

  FIG. 6 is a functional block diagram of the frame memory for explaining the averaging process of the captured image data in the calibration apparatus of the first embodiment, and FIG. 7 is a diagram showing the state transition of the switch of the frame memory in FIG. is there. Hereinafter, a case of “addition averaging processing of captured image data obtained by four consecutive photographing” will be described as an example. 6 and FIG. 7 show the addition averaging process at the time of four consecutive shootings, “the case where the shooting image data obtained by eight consecutive shootings and sixteen consecutive shootings is added and averaged. Can be dealt with by changing an addition coefficient K described later.

First, as shown in FIGS. 6 and 7, the first photographed image data (hereinafter referred to as image 01) is captured with an addition coefficient K = 0. At this time, SW1, which is a cyclic / output changeover switch, is connected to the SW102 side, so that the image 01 is stored in the frame memory FM1.
Next, the second photographed image data (hereinafter referred to as image 02) is captured with an addition coefficient K = 1/2. As a result, the image 01 read from the frame memory FM1 and the captured image 02 are added and averaged to become (image 01 + image 02) / 2, which is overwritten on the frame memory FM1.
Next, the third photographed image data (hereinafter referred to as image 03) is captured with an addition coefficient K = 2/3. As a result, the images 01 and 02 read from the frame memory FM1 and the captured image 03 are added and averaged to be (image 01 + image 02 + image 03) / 3, which is overwritten on the frame memory FM1. .
Next, the fourth photographed image data (hereinafter referred to as image 04) is captured with an addition coefficient K = 3/4. As a result, the image 01, image 02, and image 03 read from the frame memory FM1 and the captured image 04 are added and averaged to obtain (image 01 + image 02 + image 03 + image 04) / 4. At this time, as shown in FIG. 7, the circuit / output change-over switch SW1 is switched to the SW101 side, and the write FM selection switch SW2 is connected to the SW202 side, so that the averaging is performed (image 01+ Image 02 + image 03 + image 04) / 4 is stored in frame memory FM3.
Hereinafter, similarly, the addition average value of four consecutive photographed image data is alternately stored in the frame memory FM2 and the frame memory FM3.
As shown in FIG. 7, the addition average value of the four photographed image data stored in the frame memory FM2 and the frame memory FM3 in this way is the timing when the data is rewritten in one of the frame memory FM2 and the frame memory FM3. As a result, the read FM selection switch SW3 is switched, so that it is output to the outside. That is, the writing and reading of SW2 and SW3 are switched alternately.
SW1, SW2 and SW3 are described for clarifying the operation. If the writing operation and the reading operation of the frame memory are appropriately controlled, SW1, SW2 and SW3 are not necessarily provided. Also good.

Next, the operation of the display image photographing apparatus of this embodiment will be described with reference to FIGS. 1, 3 to 5, and 8. FIG.
As shown in FIG. 1, when the image projected and displayed on the screen 1 by the calibration camera 6 is continuously photographed to obtain two photographed image data, the flicker cycle and fluctuation of the illumination light source of the projector are obtained. Depending on the relationship between the period and the shutter time of the calibration camera 6, there may be a captured image deviated from “desired average brightness”.
For example, as shown in FIG. 8, when the flicker cycle of the illumination light source of the projector is 1/60 seconds and the fluctuation cycle of the projector is 1 second, the shutter time of the calibration camera 6 is less than 1/60 seconds. When the setting is short, the luminance level changes depending on the timing at which the flicker waveform is captured, so the shutter time must be an integral multiple of the projector flicker cycle. It is required to detect the fluctuation cycle.

  Therefore, in the present embodiment, the flicker cycle of the projector is detected by executing step S1 of the flowchart of FIG. 3 (that is, the flowchart of FIG. 4), and the shutter time of the calibration camera 6 is set to the flicker cycle in step S2. By controlling to an integral multiple, the influence of flicker is minimized, and exposure is adjusted so that the signal level of the photographed image does not saturate in step S3, and then step S4 (that is, the flowchart of FIG. 5) is performed. After executing the detection, the fluctuation cycle of the projector is detected and the number of times of repeated shooting corresponding to the fluctuation cycle is determined. Then, in step S5, the calibration camera 6 uses the shutter speed that is an integral multiple of the flicker cycle on the screen 1. Projection table The captured image is repeatedly photographed the number of times of repeated photographing, and the picked-up image is averaged in step S6, so that a picked-up image subjected to flicker correction and fluctuation correction is generated in step S6. Become. Therefore, the display image photographing device of the present embodiment is a display image photographing device capable of correcting both the luminance change caused by the flicker of the projector and the luminance change caused by the fluctuation of the projector, and the luminance change caused by the flicker of the projector It is possible to provide a display image photographing method capable of correcting both luminance changes caused by projector fluctuations.

  In the first embodiment, the “aperture 6c and aperture drive” is used as the incident light amount control means for limiting the incident light amount to the CCD 6c of the calibration camera 6 so that the signal level of the photographed image does not saturate. However, as shown in FIG. 9, instead of “a driving circuit 6j, a motor 9 driven by the driving circuit 6j, and rotation of the motor 9, the ND filter 10a, ND Any one of the filter 10b, the ND filter 10c, and the cavity 10d may be an “ND filter mechanism including the turret 10 disposed in front of the optical path of the CCD 6c”. The diaphragm mechanism and the ND filter mechanism may be used in combination.

  In the first embodiment, the fluctuation cycle is detected a plurality of times in step S4 of FIG. 3 to detect the fluctuation period a plurality of times, and [(fluctuation period maximum value−fluctuation period minimum value) / fluctuation period maximum value]. 3 is obtained, the number of repeated shootings is determined based on the reciprocal of the fluctuation deviation, and the main shooting of step S5 in FIG. Since the fluctuation cycle can be accurately obtained by eliminating the fluctuation, it is possible to accurately correct the luminance change caused by the fluctuation of the projector as the projection means. As a specific example, when the fluctuation cycle detection value when the fluctuation cycle detection in step S4 is performed three times is 0.9 seconds, 0.95 seconds, and 1 second, the fluctuation deviation is (1-0.9). ) /1=0.1=10%, so the reciprocal of 10 times is the number of repeated shooting.

[Second Embodiment]
FIG. 10 is a diagram showing a detailed configuration of a calibration camera, a personal computer, and a luminance sensor used in the display image photographing apparatus according to the second embodiment of the present invention. The present embodiment is obtained by adding “a change in which the luminance sensor 11 is added and the detection signal of the luminance sensor 11 is input to the luminance level detection unit 5a” of the PC 5 to the first embodiment. As the luminance sensor 11, a CCD, CMOS sensor or the like is used.

In addition to the calibration camera 6, the display image photographing apparatus of the present embodiment includes a luminance sensor 11 that detects the flicker period and the fluctuation period of the illumination light source of the projector by detecting the brightness of the illumination light source of the projector. Therefore, as in the first embodiment, “the process of FIG. 4 for detecting the flicker period of the illumination light source of the projector by comparing the signal level of the captured image data of the calibration camera 6” and “the calibration camera 6”. By comparing the signal levels of the captured image data, a part or all of “the process of FIG. 5 for detecting the fluctuation cycle of fluctuation caused by the light source of the projector and the heat of the housing” can be omitted. Therefore, by using the luminance change caused by the flicker cycle of the light source of the projector detected by the luminance sensor 11 and the luminance change caused by the fluctuation of the projector, “the process of FIG. 4 for detecting the flicker cycle of the illumination light source of the projector” And omitting a part or all of “the process of FIG. 5 for detecting the fluctuation cycle of fluctuation caused by the light source of the projector and the heat of the housing”, and optimizing the shutter speed, exposure conditions, and number of photographing of the calibration camera 6 Therefore, it is possible to shorten the “time required for a series of processes relating to flicker correction and fluctuation correction” and to speed up the calibration.
In addition, the display image photographing apparatus of the present embodiment can detect the flicker cycle and the fluctuation cycle of the illumination light source of the projector in real time regardless of the photographing by the calibration camera 6, so that the flicker even during the main photographing. There is also an advantage that correction and fluctuation correction can be supported.

[Third Embodiment]
FIG. 11 is a diagram showing a detailed configuration of a calibration camera, a personal computer, and a projector used in the display image photographing apparatus used for carrying out the display image photographing method according to the third embodiment of the present invention. In FIG. 11, for convenience of explanation, only one projector (projector 12) of the plurality of projectors is shown. The present embodiment is obtained by adding a change that replaces the internal configuration of the PC 5 to the first embodiment.
That is, the PC 5 of the present embodiment omits the luminance level detection unit 5a and the luminance level comparison unit 5c in the PC 5 of the first embodiment, and instead uses the temperature-fluctuation cycle table 5g, the elapsed time-fluctuation cycle table 5h, and the shooting conditions. This is configured by adding a determination unit 5i. In FIG. 11, the description of the color filter switching unit 5f, the color filter mechanism 8 and the screen 1 of the PC 5 is omitted for convenience of explanation. In the present embodiment, both the temperature-fluctuation cycle table 5g and the elapsed time-fluctuation cycle table 5h are provided, but only one of them may be provided.

The temperature-fluctuation cycle table 5g stores in advance the characteristics of the fluctuation cycle of the projector 12 with respect to the temperature in the housing of the projector 12. When a “signal relating to the projector temperature” is input to the PC 5 from the CPU 12b that detects the temperature by the temperature detection unit 12a in the housing of the projector 12, by referring to the temperature-fluctuation cycle table 5g using the projector temperature as a parameter, The fluctuation cycle of the projector 12 corresponding to the projector temperature can be obtained.
The elapsed time-fluctuation cycle table 5h stores in advance the characteristics of the fluctuation cycle of the projector 12 with respect to the lamp elapsed time (cumulative usage time of the lamp) of the lamp that is the illumination light source of the projector 12. When a “signal relating to lamp elapsed time (cumulative lamp usage time)” is input to the PC 5 from the CPU 12b, which monitors the driving state of the lamp (illumination light source) by the lamp driving unit 12c in the projector 12, the lamp elapsed time. By referring to the elapsed time-fluctuation cycle table 5h using as a parameter, the fluctuation cycle of the projector 12 corresponding to the lamp elapsed time can be obtained.

  The processing related to flicker correction and fluctuation correction performed in the display image photographing apparatus of the present embodiment is substantially the same as the processing of the first embodiment shown in the flowchart of FIG. 3, but the “fluctuation cycle fluctuation” in step S4 of FIG. The specific processing of “detection” is based on the above-mentioned “elapsed time data” instead of the “processing of detecting (determining) the fluctuation cycle based on actual captured image data” in FIG. 5 used in the first embodiment. A process of detecting (determining) the fluctuation period based on the reference to the elapsed time-fluctuation period table or the reference to the temperature-fluctuation period table based on the temperature data is used.

  The display image capturing apparatus according to the present embodiment includes an elapsed time-fluctuation cycle table 5g in which characteristics of a fluctuation cycle with respect to an elapsed time of the lamp of the projector 12 are stored in advance and a temperature-fluctuation in which characteristics of fluctuation cycles with respect to the temperature of the projector 12 are stored in advance. At least one of the period table 5h is created, and the fluctuation period is obtained by referring to the elapsed time-fluctuation period table 5g and the temperature-fluctuation period table 5h using at least one of the elapsed time and the temperature as a parameter at the time of shooting. Since the number of repeated shooting is determined based on the fluctuation cycle, and the image projected and displayed on the screen 1 by the calibration camera 6 using the shutter time that is an integral multiple of the flicker cycle is repeatedly shot for the number of times of the repeated shooting. In the actual measurement data of one embodiment The brute fluctuation cycle detecting step, the time elapsed by the parameters - fluctuation period table, the temperature - by replacing based on reference fluctuation table fluctuation cycle detection step, the processing speed will be able to expect.

[Fourth Embodiment]
FIG. 12 is a diagram showing a detailed configuration of a calibration camera, a personal computer, and a projector used in the display image photographing apparatus used for carrying out the display image photographing method according to the fourth embodiment of the present invention. In FIG. 12, for convenience of explanation, only one projector (projector 12) of the plurality of projectors is shown. The present embodiment is obtained by adding a change that replaces the internal configuration of the PC 5 to the first embodiment.
That is, the PC 5 of this embodiment adds a temperature-fluctuation cycle table 5g, an elapsed time-fluctuation cycle table 5h, and an imaging condition determination unit 5i similar to those of the third embodiment to the PC 5 of the first embodiment, and selects them. It is configured by adding a part 5j. In FIG. 12, the description of the color filter switching unit 5f, the color filter mechanism 8 and the screen 1 of the PC 5 is omitted for convenience of explanation. In the present embodiment, both the temperature-fluctuation cycle table 5g and the elapsed time-fluctuation cycle table 5h are provided, but only one of them may be provided.

The display image photographing device of the present embodiment is configured to be able to perform both the display image photographing method of the first embodiment and the display image photographing method of the third embodiment, and a selection unit 5j provided in the PC 5 Are variations in the captured image when the image projected and displayed on the screen 1 by the calibration camera 6 is captured a plurality of times in the display image capturing method of the first embodiment, and the display image capturing method of the third embodiment. It is configured to compare the variation in the captured image when the image projected and displayed on the screen 1 by the calibration camera 6 is captured a plurality of times, and to select the display image capturing method with the smaller variation.
Therefore, the display image capturing device of the present embodiment can minimize variations in captured images when captured multiple times. Therefore, the brightness change caused by the flicker of the projector 12 and the brightness change caused by the fluctuation of the projector 12 are achieved. Both of these can be corrected reliably.

[Fifth Embodiment]
FIG. 13 is a diagram showing a detailed configuration of a calibration camera, a personal computer, and a projector used in the display image photographing apparatus used for carrying out the display image photographing method according to the fifth embodiment of the present invention. In FIG. 12, for convenience of explanation, only one projector (projector 12) of the plurality of projectors is shown. The present embodiment is obtained by adding a change that replaces the internal configuration of the PC 5 to the first embodiment.
That is, the PC 5 of the present embodiment is configured by omitting the number-of-shoots setting unit 5e in the PC 5 of the first embodiment and adding a fluctuation cycle detection unit 5k instead. In FIG. 11, the description of the color filter switching unit 5f, the color filter mechanism 8 and the screen 1 of the PC 5 is omitted for convenience of explanation.

The display image photographing apparatus of the present embodiment is not “the photographing number setting unit 5e directly instructs the number of photographings by the calibration camera 6 to the exposure control unit 5d” as in the first embodiment. The fluctuation cycle detection unit 5k is configured to instruct the exposure control unit 5d of the fluctuation cycle as the photographing time by the calibration camera 6 ". Therefore, when an image projected and displayed on the screen 1 by the projector 12 is taken by the calibration camera 6, the flicker cycle of the projector 12 is detected, and the shutter time of the calibration camera 6 is an integer of the flicker cycle. The fluctuation period detection unit 5k detects a fluctuation period that is a luminance change caused by illumination light of the projector 12 or a luminance change caused by heat of the housing of the projector 12, and the like, and is an integral multiple of the flicker period. An image projected and displayed on the screen 1 by the calibration camera 6 using the shutter time is repeatedly shot during the fluctuation cycle, and the number of shot images corresponding to the fluctuation cycle obtained by the repeated shooting step is averaged. Will do.
Therefore, the display image photographing device of the present embodiment is a display image photographing device capable of correcting both the luminance change caused by the flicker of the projector and the luminance change caused by the fluctuation of the projector, and the luminance change caused by the flicker of the projector It is possible to provide a display image photographing method capable of correcting both luminance changes caused by projector fluctuations.

  In each of the above embodiments, the example in which the countermeasure against flicker and the countermeasure against fluctuation of the present invention are applied to the seamless multi-display device in which the images projected on the screen partially overlap has been described. Flicker that can be realized by the present invention. The countermeasure and the fluctuation countermeasure are taken by photographing the subject illuminated by `` flicker of the light source for projector illumination and other light sources with fluctuations similar to the fluctuation caused by the heat of the projector light source and the housing ''. The present invention can also be applied to an image capturing system to be acquired. In this case, it is advantageous because an image with less influence of flicker and fluctuation can be acquired.

It is a figure which shows the whole structure of the display image imaging device used for implementation of the display image imaging method of 1st Embodiment of this invention. It is a figure which shows the detailed structure of the camera for a calibration used for the display image imaging device of 1st Embodiment, and a personal computer. It is a flowchart which shows the outline of the process for the flicker correction and fluctuation correction which are implemented in the display image photographing device of the first embodiment. It is a flowchart which shows the process for the detection of the flicker period implemented in the display image imaging device of 1st Embodiment. It is a flowchart which shows the process for the detection of the fluctuation cycle implemented in the display image imaging device of 1st Embodiment. It is a functional block diagram of the frame memory for demonstrating the averaging process of the picked-up image data in the display image photographing device of 1st Embodiment. It is a figure which shows the state transition of the switch of the frame memory of FIG. It is a figure which illustrates the relationship between a flicker period and a fluctuation period, and a signal level used for description of an effect | action of the display image imaging device of 1st Embodiment. It is a figure which shows the modification of the display image imaging device of 1st Embodiment. It is a figure which shows the whole structure of the display image imaging device used for implementation of the display image imaging method of 2nd Embodiment of this invention. It is a figure which shows the detailed structure of the camera for a calibration used for the display image imaging device used for implementation of the display image imaging method of 3rd Embodiment of this invention, a personal computer, and a projector. It is a figure which shows the detailed structure of the camera for a calibration used for the display image imaging device used for implementation of the display image imaging method of 4th Embodiment of this invention, a personal computer, and a projector. It is a figure which shows the detailed structure of the camera for a calibration used for the display image imaging device used for implementation of the display image imaging method of 5th Embodiment of this invention, a personal computer, and a projector.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Screen 2 Overlapping part 3, 3a, 3b, 3c, 3d Projector 4 Image correction process part 5 Personal computer (PC)
5a Luminance level detection unit 5b Correction data calculation storage unit 5c Luminance level comparison unit 5d Exposure control unit 5e Shooting number setting unit 5f Color filter switching control unit 5g Elapsed time-fluctuation cycle table 5h Temperature-fluctuation cycle table 5i Shooting condition determination unit 5j Selection unit 5k Fluctuation period detection unit 6 Calibration camera 6a Lens mechanism 6b Aperture mechanism 6c CCD
6d S / H / AGC circuit 6e A / D conversion unit 6f Camera signal processing unit 6g Input / output I / F unit 6h CCD drive unit 6i Aperture drive unit 6j Drive circuit 8 Color filter mechanism 8a X filter 8b Y filter 8c Z filter 8d Turret 8f Cavity 9 Motor 10 Turret 10a, 10b, 10c ND filter 10d Cavity 11 Luminance sensor 12 Projector 12a Temperature detection unit 12b CPU
12c Lamp drive unit

Claims (10)

A display image photographing method for photographing an image projected and displayed on a display surface by a projecting means by a photographing means,
A flicker cycle detecting step for detecting a flicker cycle of the projection means;
A shutter time control step of controlling the shutter time of the photographing means to an integral multiple of the flicker cycle;
A fluctuation period detecting step of detecting a fluctuation period that is a luminance change caused by illumination light of the projection means or a luminance change caused by heat of a housing of the projection means, and the like;
A number-of-shooting determination step for determining the number of repeated shootings corresponding to the fluctuation cycle based on the shutter time;
A repetitive photographing step of repetitively photographing the image projected and displayed on the display surface by the photographing means using the shutter time that is an integral multiple of the flicker cycle, for the determined repetitive number of times;
A photographed image averaging step of averaging the photographed images for the number of repeated photographing obtained by the repeated photographing step;
A display image photographing method characterized by sequentially performing the steps. The flicker cycle detection step includes
A first step of photographing the image projected and displayed on the display surface by the photographing means a plurality of times using the same high shutter speed;
A second step of obtaining a level difference by comparing signal levels of a plurality of captured images taken at the same shutter speed by the photographing means;
A third step of photographing the image projected and displayed on the display surface by the photographing means using the same shutter speed that is lower than the same shutter speed a plurality of times;
A fourth step of obtaining a level difference by comparing signal levels of a plurality of captured images taken at the same low shutter speed by the photographing means;
A fifth step of comparing the previous level difference in the second step with the current level difference in the fourth step;
A sixth step of repeating the third to fifth steps by gradually shifting the shutter speed to a lower speed while the previous level difference is equal to or greater than the current level difference in the comparison of the fifth step;
A seventh step of detecting a shutter speed corresponding to the previous level difference as a flicker period of the projection means when the previous level difference is less than the current level difference in the comparison of the fifth step. The display image photographing method according to claim 1, wherein: The shutter time control step includes
The control is performed by a combination of an incident light amount control state of an incident light amount control means for restricting an incident light amount to the photographing means and a shutter time so that a signal level of a photographed image is not saturated. The display image photographing method according to 1. The fluctuation cycle detection step includes:
A step of photographing the image projected and displayed on the display surface by the photographing unit N (N = 1, 2,...) Times using the shutter time controlled to be an integral multiple of the flicker cycle by the shutter time control step. A photographing process for performing the process X and the process Y,
An averaging process for averaging each of the captured image obtained by the process X and the captured image obtained by the process Y;
A level difference calculating step of comparing the levels of the averaged captured image of the step X and the averaged captured image of the step Y to obtain a level difference;
A level difference comparison step for comparing a level difference when the number of photographing is (N-1) and a level difference when the number of photographing is N;
If the level difference when the number of times of shooting is N is smaller than the level difference when the number of times of shooting is (N−1) in the level difference comparing step, the number of times of shooting is set as (N + 1) and the level difference comparison is performed. A re-shooting process that repeats the process;
In the level difference comparison step, when the level difference when the number of times of shooting is N is greater than the level difference when the number of times of shooting is (N-1), the number of times of shooting that determines the number of times of shooting as (N-1). A decision process,
The display image photographing method according to claim 1, wherein a product of a shutter speed in the photographing step and a photographing number determined in the photographing number determining step is detected as the fluctuation cycle.   The fluctuation cycle detection step is performed a plurality of times to detect the fluctuation cycle a plurality of times, and determined based on the reciprocal of the fluctuation deviation defined by [(fluctuation cycle maximum value−fluctuation cycle minimum value) / fluctuation cycle maximum value]. The display image photographing method according to claim 1, wherein the repetitive photographing step is performed by the number of times of repeated photographing.   At least one of an elapsed time-fluctuation cycle table in which the characteristics of the fluctuation period with respect to the elapsed time of the illumination light source of the projection unit are stored in advance and a temperature-fluctuation cycle table in which the characteristics of the fluctuation cycle with respect to the temperature of the projection unit are stored in advance are created. The fluctuation period is obtained by referring to the elapsed time-fluctuation period table and the temperature-fluctuation period table using at least one of the elapsed time and the temperature as a parameter at the time of photographing. 5. The display image photographing method according to any one of 4 above.   The display image photographing method according to any one of claims 1 to 4, and the variation in the photographed image when the image projected and displayed on the display surface is photographed a plurality of times by the display image photographing method according to claim 6. 7. The display image according to claim 6, wherein the display image photographing method having the smaller variation is selected by comparing the variation of the photographed image when the image projected and displayed on the display surface is photographed a plurality of times. Shooting method. A display image photographing method for photographing an image projected and displayed on a display surface by a projecting means by a photographing means,
A flicker cycle detecting step for detecting a flicker cycle of the projection means;
A shutter time control step of controlling the shutter time of the photographing means to an integral multiple of the flicker cycle;
A fluctuation period detecting step of detecting a fluctuation period which is a luminance change caused by illumination light of the projection means or a luminance change caused by heat of the housing of the projection means by the fluctuation period detection means;
A repetitive photographing step of repetitively photographing the image projected and displayed on the display surface by the photographing means using the shutter time that is an integral multiple of the flicker period during the fluctuation period;
A photographed image averaging step of averaging the number of photographed images corresponding to the fluctuation cycle obtained by the repeated photographing step;
A display image photographing method characterized by sequentially performing the steps. A display image photographing device for photographing an image projected and displayed on a display surface by a projecting means,
Photographing means for photographing an image projected and displayed on the display surface and outputting a photographed image;
Flicker cycle detection means for detecting the flicker cycle of the projection means;
Shutter time control means for controlling the shutter time of the photographing means to an integral multiple of the flicker cycle;
A fluctuation period detecting means for detecting a fluctuation period that is a luminance change caused by illumination light of the projection means or a luminance change caused by heat of a housing of the projection means, and the like;
A number-of-shooting determination unit that determines the number of repeated shootings corresponding to the fluctuation period based on the shutter time;
Repetitive photographing means for repetitively photographing the image projected and displayed on the display surface by the photographing means using the shutter time that is an integral multiple of the flicker cycle, for the determined repetitive number of times;
Photographic image averaging means for averaging the photographic images for the number of times of repeated photographing obtained by the repeated photographing means;
A display image photographing apparatus comprising:   The display image photographing apparatus according to claim 9, wherein a sensor functioning as the flicker period detecting unit and the fluctuation period detecting unit is provided separately.

Images