JP2006135381A - Calibration method and calibration apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a calibration method and a calibration apparatus for acquiring a picked-up image to which flicker correction is applied even when a period of flicker is slower than a shutter speed. <P>SOLUTION: The calibration apparatus comprises: an exposure control section 5d for commanding a shutter speed and the diaphragm of a calibration camera 6 for photographing an image projected on a screen 1 and outputting a photographed image; a luminance level comparison section 5c for discriminating whether or not a luminance level difference between two photographed images obtained by consecutive shots at the same shutter speed is a set value or over; a photographing number of times setting section 5e for setting the number of photographing times to a plurality of number of times N when the luminance level difference does not reach a value less than the setting value when the consecutive shots are repeated while a shutter speed is gradually decreased; and a correction data calculation storage section 5b that averages photographed images for the portion of a plurality of the number of times which are obtained by consecutive shots of the image for a plurality of the number of times by the calibration camera 6 at a lower limit shutter speed so as to generate the imaged image whose flicker is corrected. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a calibration method and a calibration apparatus that perform flicker correction based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection unit. .

Conventionally, there has been proposed a multi-display device that displays an entire image by projecting divided images from a plurality of projectors onto a screen. In this multi-display apparatus, an overlap portion is provided at the end of each divided image, and an adjacent overlap portion is bonded to form one whole image. Is possible. This multi-display device is configured by using a plurality of projectors having the same specification, but in addition to geometrical deviation caused by positional accuracy when the projectors are arranged, variations in projector components and brightness of the illumination lamp Due to variations in color, color variations, brightness (luminance) variations, white balance differences, and the like may occur. In that case, when displaying the whole image by overlapping multiple divided images and pasting them together, the connected part becomes conspicuous due to the variation in color and brightness between the divided images, and the display quality is impaired. End up. Therefore, in such a conventional multi-display device, a calibration camera (for example, a digital camera) is installed in front of the screen, an image (for example, a test pattern image) displayed on the screen is captured, and the captured image data is converted into captured image data. Based on this, correction data (for example, a profile indicating the color characteristics of each projector) is generated, and when a partial image is projected from each projector, color correction and geometric correction are performed with reference to each profile. .
As another conventional technique for creating a profile, there is a system in which a test pattern image is projected on a screen and photographed with a digital camera or the like, and a profile is created from photographed data. In the following, taking a test pattern image with a digital camera and creating a profile is referred to as “calibration”, the camera used for this purpose is referred to as “calibration camera”, and the system used for this is referred to as “calibration”. Will be referred to as a “system”.

  When acquiring color correction data in the calibration system applied to the multi-display device, a test pattern is sent from each PC to each projector, and tests with different brightness and color are performed on the screen from each projector. When the pattern is projected in time series, the test pattern displayed on the screen is photographed by the calibration camera, the photographed data is stored in the PC, and the correction data is calculated in the PC. The correction data obtained in this way is sent to the image correction processing unit, where the image data used for actual display is corrected in real time and sent to each projector. Therefore, by projecting the image incorporating the correction, Then, an image whose color and brightness are corrected is displayed on the screen. The color correction includes gamma correction for each small area or pixel for reducing color unevenness between projectors (or color unevenness within each projector), R (Red), G (Green), and B ( Blue) There is white balance correction for matching the display level of each color.

When acquiring the gamma correction data in the calibration system, after setting the shutter speed in the calibration camera (for example, a digital camera) so that the luminance level is in an appropriate range, the minimum level (black) in each RGB color After projecting a plurality of images (test patterns) in increments of a predetermined luminance value from the first to the highest level onto the screen, each image is photographed, and after detecting the luminance level on the PC, the luminance level difference of each projector Is calculated, and correction data that eliminates the brightness level difference between the projectors is created.
On the other hand, when acquiring white balance correction data, a shutter speed that sets the luminance level within an appropriate range is set in a calibration camera (for example, a digital camera), and then, for example, CIE 1931 is used on the calibration camera side. XYZ color system A filter that passes through the X, Y, and Z wavelength regions of the color matching function, with an X filter, a Y filter, and a Z filter sequentially inserted in the camera optical path, an image (test) As a pattern), a test pattern of R brightness value 255 (all red), a test pattern of G brightness value 255 (all green), and a test pattern of B brightness value 255 (all blue) are sequentially projected on the screen, Each image is taken, and correction data is created so that each color is balanced on the PC.

  When shooting a plurality of images (test patterns) in order to obtain gamma correction data and white balance correction data in the calibration system described above, exposure such as shutter speed adjustment is performed in a calibration camera (for example, a digital camera). It is necessary to control. However, a discharge tube type light source using discharge between electrodes, such as an ultra-high pressure mercury lamp or a xenon lamp, which is generally used as an illumination light source for a projector, has a flicker that is a change in the light amount of the light source at a substantially constant period. The fluctuation of the intensity of the illumination light projected from such a light source is unstable. It becomes. Further, the brightness of the projected image changes in an unstable manner due to the fluctuation of the air on the optical path caused by the heat generated from the projector housing. Therefore, if an image projected on such a screen is shot at a shutter speed faster than the flicker of the projector light source, the luminance becomes unstable because “flicker cycle> shutter speed”. An image deviated from “desired average brightness” is taken. More specifically, when flickering occurs when shooting while gradually changing the brightness from black to white, accurate brightness change data cannot be acquired, so the brightness level between projectors does not match. Occurs.

  As a conventional technique for dealing with flicker (fluctuation), the frequency analysis means performs frequency analysis on the detection output of the illumination light detection means for detecting a change in the illumination light of the subject, and the cycle of the most frequent frequency component in the output of the frequency analysis means Is calculated as an exposure time, and the exposure time is given to an imaging device that captures an optical image of the subject (see, for example, Patent Document 1, hereinafter referred to as Conventional Technology 1), the first image signal, Amplifying means for amplifying one image signal and outputting a second image signal; a first signal level obtained by sampling the first image signal at a first period; and a second image signal from the first period. Gain control means for controlling the gain of the amplifying means based on a second signal level sampled in a short second period, and the second period is used for taking an image of the first image signal. Is an image signal fluctuation compensating circuit corresponding to the period of fluctuation of the radiation intensity of the illumination light sources (for example, see Patent Document 2, hereinafter referred to as prior art 2) it is. In addition to the above, as a basic flicker countermeasure, there is “matching the shutter speed to the flicker cycle”.

JP-A-7-336586 JP 2004-141382 A

Since the prior art 1 is a measure assuming that the cause of flicker is a relatively high frequency and a constant frequency such as a fluorescent lamp (100 Hz), it cannot cope with a case where the flicker cycle is slower (longer) than the shutter speed. There is a problem.
In the prior art 2, the gain of the amplifying means is controlled (increased) instead of changing the shutter speed in accordance with the flicker cycle, so that the S / N ratio of the image signal deteriorates as the gain increases. There is a problem that.

It is a first object of the present invention to provide a calibration method capable of acquiring a picked-up image subjected to flicker correction even when the flicker cycle of the light source of the projection unit is slower (longer) than the shutter speed of the photographing unit. To do.
It is a second object of the present invention to provide a calibration device that can acquire a flicker-corrected captured image even when the flicker cycle of the light source of the projection unit is slower (longer) than the shutter speed of the imaging unit. To do.

  In order to achieve the first object, the first invention described in claim 1 is based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection means. A calibration method for performing flicker correction by obtaining a luminance level difference between two captured images obtained when the image is continuously captured at the same shutter speed, and determining whether the luminance level difference is equal to or greater than a set value. A first step of determining; a second step of setting a low-speed shutter speed that is slower than the same shutter speed while the brightness level difference is equal to or greater than the set value; and the low-speed shutter until the lower limit shutter speed is reached. A third step that repeats the first step and the second step using a speed, and the image using the lower limit shutter speed when the lower limit shutter speed is reached. A fourth step of continuously shooting a plurality of times at the same shutter speed, and a fifth step of generating a picked-up image corrected by flicker by averaging the plurality of shot images obtained in the fourth step. It is characterized by performing sequentially.

  In order to achieve the first object, the second invention according to claim 2 is based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection means. A calibration method for performing flicker correction by obtaining a luminance level difference between two captured images obtained when the image is continuously captured at the same shutter speed, and determining whether the luminance level difference is equal to or greater than a set value. A first step of determining; a second step of setting a low-speed shutter speed that is slower than the same shutter speed while the brightness level difference is equal to or greater than the set value; and the low-speed shutter until the lower limit shutter speed is reached. A third step that repeats the first step and the second step using a speed, and when the shutter speed is reached, a luminance level difference is determined according to the luminance level difference. A fourth step of setting the number of shootings to a plurality of times so as to increase the number of times, a fifth step of continuously shooting the image a plurality of times at the same shutter speed using the lower limit shutter speed, and the fifth step And a sixth step of generating a picked-up image that has been subjected to flicker correction by averaging the picked-up images for a plurality of times obtained by the above.

  In order to achieve the second object, a third invention according to claim 3 is based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection means. A calibration device that performs flicker correction, photographing the image and outputting the photographed image, exposure control means commanding the shutter speed and aperture of the photographing means, and the same shutter speed by the photographing means Brightness level comparison means for determining whether or not the difference in brightness level between two captured images obtained when continuously shooting with a camera is greater than or equal to a set value, and the continuous shooting while gradually decreasing the shutter speed of the shooting means When the brightness level difference does not become less than the set value when repeating the above, the number-of-shooting setting means for setting the number of shots to a plurality of times, and the exposure control Correction data generating means for generating a picked-up image that has been subjected to flicker correction by averaging the plurality of captured images obtained when the image is continuously photographed a plurality of times by the photographing means at a lower limit shutter speed commanded by the means It is characterized by comprising.

  The fourth aspect of the present invention is characterized in that the photographing number setting means sets the plurality of times so that the number of times increases as the luminance level difference increases.

  According to a fifth aspect of the present invention, a sensor for detecting a flicker period of a light source of the projection unit is provided separately.

  According to the first invention, in order to adjust the display state of the image projected on the display surface by the projecting means, when performing flicker correction based on the photographed image obtained by photographing the image, the image is taken as the same shutter. A first step of obtaining a luminance level difference between two captured images obtained at the time of continuous shooting at a speed and determining whether the luminance level difference is greater than or equal to a set value; and the luminance level difference is greater than or equal to the set value For a while, the second step of setting a low-speed shutter speed that is slower than the same shutter speed is repeated using the low-speed shutter speed until the lower-limit shutter speed is reached, and when the lower-limit shutter speed is reached, By continuously shooting the image a plurality of times at the same shutter speed using a lower limit shutter speed, and averaging the captured images obtained by the plurality of times. Since a captured image with flicker correction is generated, a calibration method capable of acquiring a captured image with flicker correction even when the flicker cycle of the light source of the projection unit is slower (longer) than the shutter speed of the imaging unit. Can be provided.

  According to the second invention, in order to adjust the display state of the image projected on the display surface by the projecting means, when performing flicker correction based on the photographed image obtained by photographing the image, the image is subjected to the same shutter. A first step of obtaining a luminance level difference between two captured images obtained at the time of continuous shooting at a speed and determining whether the luminance level difference is greater than or equal to a set value; and the luminance level difference is greater than or equal to the set value For a while, the second step of setting a low-speed shutter speed slower than the same shutter speed is repeated using the low-speed shutter speed until the lower-limit shutter speed is reached. According to the level difference, after setting the number of times of photographing to a plurality of times so that the number of times is increased as the luminance level difference is larger, the same image is made using the lower limit shutter speed. By capturing the plurality of times continuously at the shutter speed and averaging the plurality of captured images obtained thereby, a picked-up image that has been subjected to flicker correction is generated based on the plurality of times with the number of times optimized. Therefore, it is possible to provide a calibration method capable of acquiring a picked-up image that has been subjected to flicker correction according to the brightness level difference even when the flicker cycle of the light source of the projection unit is slower (longer) than the shutter speed of the photographing unit. Can do.

  According to the third invention, the calibration device that performs flicker correction based on the photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projecting unit, photographs the image. And a brightness level between two captured images obtained when continuous photographing is performed at the same shutter speed by the photographing means, a photographing means for outputting a photographed image, an exposure control means for instructing a shutter speed and an aperture of the photographing means, A brightness level comparison unit that determines whether or not the difference is greater than or equal to a set value, and the brightness level difference is less than the set value when the continuous shooting is repeated while gradually decreasing the shutter speed of the shooting unit. If not, the shooting number setting means for setting the number of shooting times to a plurality of times, and the lower limit shutter speed commanded by the exposure control means, Correction data generating means for generating a picked-up image that has been subjected to flicker correction by averaging the plurality of captured images obtained when the image is continuously photographed a plurality of times by the projection means. Even if the flicker cycle of the light source is slower (longer) than the shutter speed of the photographing means, it is possible to provide a calibration device capable of acquiring a picked-up image subjected to flicker correction.

  According to the fourth invention, the number of times of photographing by the photographing means is optimized because the number of times of photographing is set so that the number of times increases as the luminance level difference increases. As described above, the influence of the flicker of the light source of the projection means can be reduced.

  According to the fifth aspect of the present invention, since the sensor for detecting the flicker cycle of the light source of the projection means is provided separately, when it is desired to speed up the process of generating the picked-up image corrected in the third aspect, the sensor By using the flicker cycle of the light source of the projection means detected by the above, it becomes possible to optimize the number of times of photographing and exposure of the photographing means, thereby shortening the time required for flicker correction and speeding up the calibration can do.

  Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing the overall configuration of a calibration apparatus used for carrying out the calibration method of the first embodiment of the present invention, and FIG. 2 is a calibration camera and personal computer used in the calibration apparatus of the first embodiment. FIG.

  The calibration device according to the present embodiment captures the image when performing color correction (gamma correction and white balance correction) in order to adjust the display state of the image projected on the screen of the multi-display device. , And a plurality of (four in the illustrated example) projection means for projecting the divided images so as to have an overlap portion 2 on the screen 1 as shown in FIG. Projectors 3a, 3b, 3c, and 3d, an image correction processing unit 4 that supplies divided image signals to the projectors 3a, 3b, 3c, and 3d, and projectors 3a, 3b, A plurality of images (hereinafter also referred to as test patterns) for correcting variations in the image display state between 3c and 3d are displayed on the screen 1. A personal computer (PC) 5 having an image projecting function for sequentially projecting and a flicker correction function for generating captured image data subjected to flicker correction by continuously capturing an image displayed on the screen 1 as described later, a plurality of times; A calibration camera 6 which is a photographing means for photographing the plurality of images under a predetermined exposure condition defined by a shutter speed and an aperture, and a color filter provided at a predetermined position on the same optical path as the calibration camera 6 And a mechanism 8.

  The image correction processing unit 4 includes a correction data storage unit 4a for storing correction data created by the PC 5, and divided image data (which may be supplied by the image correction processing unit 4 itself). And a correction processing unit 4b that outputs the corrected data by adding the correction by the correction data.

  As shown in FIG. 2, the PC 5 includes a luminance level detection unit 5 a that detects a luminance level of “photographed image data of an image projected on the screen 1” input from the calibration camera 6, and the “screen”. On the basis of “photographed image data of an image projected on 1”, correction data relating to the projector that projected the image is calculated (generated) and stored, and the “image projected on the screen 1 is continuously photographed a plurality of times” When continuous shooting is performed at the same shutter speed by the correction data calculation storage unit 5b that calculates (generates) and stores correction data (flicker corrected captured image data) based on the “image data” and the calibration camera 6. Compare the brightness levels of the two captured images and if the brightness level difference is greater than or equal to the set value A luminance level comparison unit 5c that determines whether or not the exposure is performed, an exposure control unit 5d that performs exposure control (shutter speed command, aperture command) of the calibration camera 6 based on the determination result of the luminance level comparison unit 5c, and luminance Based on the brightness level detected by the brightness level detection unit 5a and the number-of-shoots setting unit 5e that sets (commands) the number of times of shooting at the time of multiple times shooting, which will be described later, to the calibration camera 6 based on the determination result of the level comparison means 5c. And a color filter switching control unit 5f that selectively switches color filters used when sequentially taking the plurality of images. Note that the color filter control unit 5f performs CIE 1931 XYZ color system color matching function X during white balance correction, as will be described later, during a series of imaging periods of the plurality of test pattern images by the calibration camera 6. , Y, and Z, which are filters that pass through the wavelength regions of Y, Z, and Z, are sequentially switched and used, and filter switching control that uses a cavity without a color filter is performed during gamma correction.

  As the calibration camera 6, a monochrome digital camera which is a monochrome photographing unit is used in the present embodiment. As shown in FIG. 2, the calibration camera 6 includes a lens mechanism 6a that forms an image of a subject, a diaphragm mechanism 6b that adjusts the amount of light emitted from the lens mechanism 6a, and a subject that is incident through the diaphragm mechanism 6b. A CCD 6c that photoelectrically converts the image of the image into an electrical signal (image data signal), an S / H • AGC circuit 6d that performs sample hold and automatic gain control (AGC) on the image data signal output from the CCD 6c, An A / D converter 6e for A / D converting the image data signal output from the H • AGC circuit 6d, a camera signal processor 6f for performing various signal processing on the A / D converted image data signal, and a camera An input / output I / F unit 6g that outputs an image data signal output from the signal processing unit 6f to the PC 5 and relays various signals input from the PC 5, A CCD drive unit 6h that drives the CCD 6c at the shutter speed commanded from the light control unit 5d, and an aperture drive unit 6i that drives the aperture mechanism 6b so that the aperture value commanded from the exposure control unit 5d of the PC 5 is obtained. It comprises. In the present embodiment, a camera capable of setting the shutter speed to, for example, 1/1000 sec to 2 sec is used as the calibration camera 6. As the calibration camera 6, in addition to a monochrome digital camera, a monochrome video camera, a monochrome CMOS sensor, a monochrome line sensor, or the like, which is a monochrome photographing unit, can also be used.

The color filter mechanism 8 is used when photographing a test pattern for white balance correction, and is an X filter 8a that is a filter that passes through the X, Y, and Z wavelength regions of the CIE 1931 XYZ color system color matching function. , Y filter 8b, Z filter 8c, a turret 8d having four openings for mounting these filters on the circumference, and a filter (or cavity) corresponding to a command from the color filter switching control unit 5f of the PC 5 ) Is a motor (not shown) that drives the turret 8c to rotate around the central axis so that it is located on the optical path. Used when shooting images for gamma correction that are not used.
In the color filter mechanism 8, the CIE 1931 XYZ color system color matching function is used as a plurality of color filter means capable of separating the input light of the calibration camera 6 into a plurality of wavelength characteristics for white balance correction. X filters, Y filters, and Z filters, which are filters that pass through each wavelength region of X, Y, and Z, are used, but any color filter can be used even if R, G, and B pass filters are used instead. It may be used.

Next, calibration performed in the calibration apparatus of the present embodiment will be described. In the calibration apparatus of this embodiment, color correction and geometric correction are performed as calibration, and the procedure is as follows.
First, as shown in FIG. 1, the calibration camera 6 is arranged at the observation position of the general audience. Next, a test pattern image for geometric correction is projected on the screen 1 and photographed, and geometric correction data is created by the PC 5 based on the photographed data (in the case of a multi-display device in which geometric deviation does not occur, geometric correction is Unnecessary).
Thereafter, a test pattern image for color correction (gamma correction and white balance correction) with geometric correction is projected onto the screen 1 and photographed, and color correction data is created by the PC 5 based on the photographed data. At this time, if the flicker cycle of the light sources of the projectors 3a, 3b, 3c, and 3d, which are projection means, is slower (longer) than the shutter speed of the calibration camera 6, shooting is performed by the calibration camera 6 at the shooting timing. Considering that the brightness level of the captured image data fluctuates, “flicker correction” which is the target of the present invention is performed.

  In the color correction performed as described above, small area or pixel-by-pixel gamma correction for reducing color unevenness between projectors (or color unevenness in each projector), R (Red), G White balance correction is performed to match the display levels of the (Green) and B (Blue) colors. The small area to be corrected in gamma correction can be determined according to the number of pixels of the calibration camera (digital camera) to be used. For example, the number of display pixels on the screen 1 is 8 million. When the number of pixels of the digital camera is 2 million pixels, every 2 × 2 pixels is a correction area. In the gamma correction, a plurality of test pattern images having a predetermined luminance value in a range from the lowest level (black) to the highest level of the luminance signal is projected on the screen 1 for each color of RGB, and the test pattern image is projected by the calibration camera 6. Take a picture. On the other hand, the white balance correction is based on human visual sensitivity such as CIE 1931 XYZ color system color matching function by projecting a test pattern image of appropriate luminance values (for example, maximum luminance values) of RGB colors onto the screen 1. A test pattern image is photographed by the calibration camera 6 through the filter, and correction is performed so that each color is balanced based on the photographing data.

Next, the flicker correction performed in the calibration apparatus of the present embodiment will be described in detail with reference to FIGS.
FIG. 3 is a flowchart showing flicker correction in the calibration apparatus of the first embodiment. Note that the flicker correction of FIG. 3 is performed at the same time when the test pattern image for color correction is captured, but may be performed prior to the capture of the test pattern image for color correction. .
First, in step S1 of FIG. 3, the shutter speed of the calibration camera 6 is set to a predetermined shutter speed. As this predetermined shutter speed, in this embodiment, a high-speed shutter speed (for example, 4 msec) that can be set in the calibration camera 6 is used. In the next step S2, images projected on the screen 1 by the projector 3a or the like are continuously photographed at the shutter speed, and two images are stored in a frame memory (see FIG. 4 described later) in the correction data calculation storage unit 5b of the PC 5. Capture captured image data. In the next step S3, the two photographed image data are integrated to obtain respective luminance levels, and the difference between the two luminance levels is calculated. In the next step S4, it is determined whether or not the luminance level difference is greater than or equal to a set value (for example, 5%). If this determination is NO, the process proceeds to a step S5, and if YES, the process proceeds to S6. In the above description, the shutter speed is 4 msec and the setting value used for the determination in step S4 is 5%. However, the present invention is not limited to this, and both may be arbitrary values.

  In step S5 in which the process proceeds when the brightness level difference is less than a set value (for example, 5%), the shooting condition is set to “step” in consideration that the flicker of the projector light source hardly affects the brightness level of the captured image data. "Shoot once at the shutter speed used for continuous shooting in S2." Then, in the next step S7, the test pattern image for color correction is shot (hereinafter referred to as main shooting) under the shooting conditions determined as described above.

  On the other hand, when the brightness level difference is equal to or greater than a set value (for example, 5%), the process proceeds in step S6, where the flicker of the projector light source affects the brightness level of the captured image data. In view of the concern that the accuracy of correction data for color correction may be reduced, it is determined whether or not the shutter speed can be reduced. If the shutter speed can be decreased in this determination, after the shutter speed is decreased (for example, decreased by one level) in the next step S8, the process is returned to step S2 and step S2 and subsequent steps are repeated.

  In step S2 for the second time, continuous shooting is performed at a shutter speed one step lower than the first time (for example, 8 msec if the first time is 4 msec), and two shot image data are stored in the frame memory in the correction data calculation storage unit 5b of the PC5. take in. In the next step S3, the two photographed image data are integrated to obtain the respective luminance levels, and the difference between the two luminance levels is calculated. In the next step S4, it is determined again whether the luminance level difference is a set value (for example, 5%) or more. If this determination is NO, the process proceeds to a step S5, and if YES, the process proceeds to S6. Similarly, the continuous shooting is repeated while gradually decreasing the shutter speed of the calibration camera 6 until the determination in step S4 becomes NO, but the lower limit shutter speed commanded by the exposure control unit 5d of the PC 5 (for example, If the determination in step S4 does not become NO even when the shutter speed is reduced to 2 msec) (that is, in the case of NO in step S6), the flicker correction of the present invention is necessary, so step S9 The number of shooting is set to a plurality of N times (for example, 4 times). Then, in the next step S10, the shooting condition is determined as “shooting multiple times N times at the lower limit shutter speed”. In the next step S11, a test pattern image for color correction is shot (hereinafter referred to as main shooting) under the shooting conditions determined as described above. In the next step S12, picked-up image data that has been subjected to flicker correction is generated by averaging the picked-up image data for a plurality of N times obtained by continuously photographing the image N times.

  FIG. 4 is a functional block diagram of the frame memory for explaining the averaging process of the captured image data in the flicker correction in the calibration device of the first embodiment, and FIG. 5 shows the state transition of the switch of the frame memory in FIG. FIG. Hereinafter, a case of “addition averaging processing of captured image data obtained by four consecutive photographing” will be described as an example. 4 and FIG. 5 show the addition averaging process at the time of four consecutive shootings, but “addition averaging process is performed on the captured image data obtained by eight consecutive shootings and 16 consecutive shootings” Can be dealt with by changing the addition coefficient K.

First, as shown in FIGS. 4 and 5, the first photographed image data (hereinafter referred to as image 01) is captured with an addition coefficient K = 0. At this time, SW1, which is a cyclic / output changeover switch, is connected to the SW102 side, so that the image 01 is stored in the frame memory FM1.
Next, the second photographed image data (hereinafter referred to as image 02) is captured with an addition coefficient K = 1/2. As a result, the image 01 read from the frame memory FM1 and the captured image 02 are added and averaged to become (image 01 + image 02) / 2, which is overwritten on the frame memory FM1.
Next, the third photographed image data (hereinafter referred to as image 03) is captured with an addition coefficient K = 2/3. As a result, the images 01 and 02 read from the frame memory FM1 and the captured image 03 are added and averaged to be (image 01 + image 02 + image 03) / 3, which is overwritten on the frame memory FM1. .
Next, the fourth photographed image data (hereinafter referred to as image 04) is captured with an addition coefficient K = 3/4. As a result, the image 01, image 02, and image 03 read from the frame memory FM1 and the captured image 04 are added and averaged to obtain (image 01 + image 02 + image 03 + image 04) / 4. At this time, as shown in FIG. 5, the cycle / output change-over switch SW1 is switched to the SW101 side, and the write FM selection switch SW2 is connected to the SW202 side, so that the averaging is performed (image 01+ Image 02 + image 03 + image 04) / 4 is stored in frame memory FM3.
Hereinafter, similarly, the addition average value of four consecutive photographed image data is alternately stored in the frame memory FM2 and the frame memory FM3.
As shown in FIG. 5, the addition average value of the four photographed image data stored in the frame memory FM2 and the frame memory FM3 in this way is the timing when the data is rewritten in one of the frame memory FM2 and the frame memory FM3. As a result, the read FM selection switch SW3 is switched, so that it is output to the outside. That is, the writing and reading of SW2 and SW3 are switched alternately.
SW1, SW2 and SW3 are described for clarifying the operation. If the writing operation and the reading operation of the frame memory are appropriately controlled, SW1, SW2 and SW3 are not necessarily provided. Also good.

Next, the operation of the calibration apparatus according to the present embodiment will be described with reference to FIG. 1, FIG. 3, FIG. 6, and FIG.
As shown in FIG. 1, when continuously capturing images projected on the screen 1 by the calibration camera 6 to obtain captured image data for two images, the light quantity of the illumination light source of the projector is substantially constant. Depending on the relationship between the flicker cycle, which is a change, and the shutter speed of the calibration camera 6, there may be a photographed image deviated from “desired average brightness”.
For example, as shown in FIG. 6A, when the flicker cycle of the illumination light source of the projector is 10 msec and the shutter speed of the calibration camera 6 is set to 4 msec, shooting is performed at any timing of the flicker waveform. Therefore, the brightness levels P1 and P2 of the respective captured images obtained by continuous shooting at the timing shown in the figure are P1> P2, and the brightness level difference is 5% or more. Therefore, continuous shooting is performed again with the shutter speed set to one stage by executing step S8 in FIG. In this case, as shown in FIG. 6B, when the flicker cycle of the illumination light source of the projector is 10 msec, the shutter speed of the calibration camera 6 is set to 8 msec. Since the brightness levels P1 and P2 of each captured image obtained by shooting satisfy P1> P2 and the brightness level difference is 5% or more, the shutter speed is again set to one stage by executing step S8 in FIG. Perform continuous shooting again. In this case, as shown in FIG. 6C, when the flicker cycle of the illumination light source of the projector is 10 msec, the shutter speed of the calibration camera 6 is set to 16 msec. Since the brightness levels P1 and P2 of each captured image obtained by shooting are P1 = P2 and the brightness level difference is less than 5%, the process proceeds from NO in step S4 in FIG. 3 to step S5 and subsequent steps. Thus, one actual photographing is performed at a shutter speed of 16 msec.

  On the other hand, as shown in FIG. 7, for example, when the shutter speed of the calibration camera 6 is set to 2 sec which is the lower limit shutter speed when the flicker cycle of the illumination light source of the projector is 2.5 sec, The brightness levels P1 and P2 of the respective captured images obtained by continuous shooting at such timings are P1> P2, the brightness level difference is 5% to 10%, and the shutter speed can be further reduced. Therefore, the process proceeds from NO in step S6 of FIG. 3 to step S9, and the number of times of photographing is set to a plurality of N times (for example, 4 times). After performing the main shooting by the continuous shooting of step S11, the execution of step S11 performs averaging processing on the shot image data for N times. That.

  According to the calibration device of the present embodiment, when the flicker cycle of the illumination light source of the projector is slower (longer) than the lower limit shutter speed of the calibration camera 6, the captured image data for N times are averaged. As a result, a flicker-corrected captured image is generated, so that flicker correction is performed even when the flicker cycle of the illumination light source of the projector is slower (longer) than the lower limit shutter speed of the calibration camera 6 (in other words, It is possible to provide a calibration method and a calibration apparatus capable of acquiring a captured image (in which the influence of flicker of the illumination light source of the projector is removed). Therefore, as shown in FIG. 1, the calibration of the illumination light source of the projector is performed as in the case of calibration with respect to the “multi-display device that projects the divided image on the screen 1 so as to have the overlap portion 2 from the projector”. It is possible to acquire image data that is less affected by flicker or fluctuation when shooting a subject that is illuminated by a light source with flicker that is a light amount change with a substantially constant period or a light source with fluctuation that is a slightly unstable light amount change. This is advantageous.

  In the first embodiment, a plurality of N times of continuous shooting is set to four times, but any number of times may be set. In the first embodiment, the main photographing is performed once when the brightness level difference is less than 5% while the shutter speed is lowered. However, the main photographing may be performed a plurality of times (for example, twice).

[Second Embodiment]
FIG. 8 is a flowchart showing flicker correction in the calibration apparatus according to the second embodiment of the present invention. In the present embodiment, in order to improve the flicker correction accuracy with respect to the first embodiment, “the number of continuous shootings is changed stepwise according to the brightness level difference between the two shot images. (Change to increase) ”.

  That is, in the flowchart of FIG. 8, steps S1 to S8 perform the same processing as in the first embodiment, step S1-step S2-step S3-step S4 YES-step S6 YES-step S8-step S2. By repeating this loop, while the brightness level difference between two captured images obtained when images on the screen 1 are continuously captured at the same shutter speed is equal to or higher than a set value (for example, 5%), the lower limit shutter speed is set. Continuous shooting is repeated with the shutter speed lowered until the brightness reaches the same level. When the brightness level difference becomes less than a set value (for example, 5%) during that time, the actual shooting is performed at the shutter speed by executing Step S5 to Step S7 (however, In the embodiment, the number of times of actual photographing is changed from once to twice in the first embodiment), If not become less than the luminance level difference set value (e.g., 5%), the process proceeds from NO in step S6 to step S21.

  In step S21, it is determined whether the luminance level difference belongs to “5% or more and less than 10%”, “10% or more and less than 20%”, or “20% or more”, and “5% or more and less than 10%”. If so, the process proceeds to step S22 to set N = 4 for the number of shootings to N = 4. If “10% or more and less than 20%”, the process proceeds to step S23 to set N for the number of shootings to N. If “20% or more” is set, the process proceeds to step S24, and N = 16 is set for a plurality of N times of photographing. Then, in step S25 following step S22, step S23, and step S24, a plurality of N main photographings are performed at the lower limit shutter speed, and in a next step S26, a plurality of N times obtained by the plurality of N consecutive photographings. Captured image data that has been subjected to flicker correction is generated by averaging the captured image data. Note that the number N of the number of shootings is not limited to N = 4, 8, and 16, but can be any number.

  According to the calibration device of the present embodiment, when the flicker cycle of the illumination light source of the projector is slower (longer) than the lower limit shutter speed of the calibration camera 6, the brightness level difference between the two photographed images is 5. % = 10 for less than 10%, N = 8 for 10% or more and less than 20%, and N = 16 for 20% or more. As a result, a picked-up image that is flicker-corrected based on a plurality of N shots in which the number of shots is optimized is generated, so that the flicker cycle of the illumination light source of the projector is slower than the lower limit shutter speed of the calibration camera 6 ( Even if it is long, flicker correction was performed according to the brightness level difference (in other words, the influence of flicker of the illumination light source of the projector) It is possible to provide a suitably removed) captured images that the possible acquisition, calibration method and calibration device. Therefore, as shown in FIG. 1, the calibration of the illumination light source of the projector is performed as in the case of calibration with respect to the “multi-display device that projects the divided image on the screen 1 so as to have the overlap portion 2 from the projector”. It is possible to acquire image data that is less affected by flicker or fluctuation when shooting a subject that is illuminated by a light source that has flicker with a light amount change of almost constant period or a light source with fluctuation that has a slightly unstable light quantity change. This is advantageous.

[Third Embodiment]
FIG. 9 is a diagram showing a detailed configuration of a calibration camera, a personal computer, and a luminance sensor used in the calibration apparatus according to the third embodiment of the present invention. The present embodiment is obtained by adding “a change in which the luminance sensor 9 is added and the detection signal of the luminance sensor 9 is input to the luminance level detection unit 5a” of the PC 5 to the first embodiment. As the luminance sensor 9, a CCD, CMOS sensor or the like is used.

Since the calibration apparatus according to the present embodiment includes the brightness sensor 9 that detects the flicker period of the illumination light source by detecting the brightness of the illumination light source of the projector separately from the calibration camera 6, Similar to the embodiment, “the shutter speed and the number of photographing times of the calibration camera 6 optimized with respect to the flicker cycle of the illumination light source of the projector by comparing the luminance levels of the two photographed image data of the calibration camera 6. A part or all of “the process of setting“ can be omitted ”, and the“ time required for a series of processes related to flicker correction ”can be shortened to speed up the calibration.
In addition, the calibration apparatus according to the present embodiment can detect the flicker cycle of the illumination light source of the projector in real time regardless of the shooting by the calibration camera 6, and therefore can cope with the flicker correction even during the main shooting. There is also an advantage that.

In each of the above embodiments, it is assumed that the flicker light source is an illumination light source for a projector. However, a possible flicker countermeasure in the present invention is “flicker that is a change in the light amount of the illumination light source at a substantially constant period” The present invention can also be applied to a case where a subject illuminated by a “other light source having a fluctuation with a slightly unstable light amount change” is photographed.
In the first embodiment, when the brightness level difference is less than a set value (for example, 5%) at a shutter speed faster than the lower limit shutter speed, the number of times of actual photographing is set to one. Even in this case, if the luminance level is not constant (if the flicker light source has an unstable flicker cycle), the captured image data obtained by performing a plurality of main photographings may be averaged.

It is a figure which shows the whole structure of the calibration apparatus used for implementation of the calibration method of 1st Embodiment of this invention. It is a figure which shows the detailed structure of the camera for calibration and personal computer used for the calibration apparatus of 1st Embodiment. It is a flowchart which shows the flicker correction | amendment in the calibration apparatus of 1st Embodiment. It is a functional block diagram of a frame memory for explaining averaging processing of photographed image data in flicker correction in the calibration device of the first embodiment. FIG. 5 is a diagram showing a state transition of a switch of the frame memory in FIG. 4. (A), (b), and (c) respectively, when the flicker cycle of the projector light source is 10 msec in the calibration apparatus of the first embodiment, the shutter speed of the calibration camera 6 is set to 4 msec, 8 msec, and 16 msec. It is a figure for demonstrating the brightness level difference at the time of setting. In the calibration apparatus of the first embodiment, when the flicker cycle of the projector light source is 2.5 sec, the difference in luminance level when the shutter speed of the calibration camera 6 is set to 2 sec, which is the lower limit shutter speed, will be described. FIG. It is a flowchart which shows the flicker correction | amendment in the calibration apparatus of 2nd Embodiment of this invention. It is a figure which shows the detailed structure of the camera for a calibration used for the calibration apparatus of 3rd Embodiment of this invention, a personal computer, and a brightness | luminance sensor.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Screen 2 Overlapping part 3a, 3b, 3c, 3d Projector 4 Image correction process part 5 Personal computer (PC)
5a Luminance level detection unit 5b Correction data calculation storage unit 5c Luminance level comparison unit 5d Exposure control unit 5e Shooting number setting unit 5f Color filter switching control unit 6 Calibration camera 6a Lens mechanism 6b Aperture mechanism 6c CCD
6d S / H / AGC circuit 6e A / D conversion unit 6f Camera signal processing unit 6g Input / output I / F unit 6h CCD drive unit 6i Aperture drive unit 8 Color filter mechanism 8a X filter 8b Y filter 8c Z filter 8d Turret 8f Cavity 9 Brightness sensor

Claims (5)

A calibration method for performing flicker correction based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection unit,
A first step of determining a luminance level difference between two captured images obtained when the image is continuously captured at the same shutter speed, and determining whether the luminance level difference is a set value or more;
A second step of setting a lower shutter speed that is slower than the same shutter speed while the brightness level difference is equal to or greater than the set value;
A third step of repeating the first step and the second step using the low-speed shutter speed until the lower limit shutter speed is reached;
A fourth step of continuously shooting the image a plurality of times at the same shutter speed using the lower limit shutter speed when the lower limit shutter speed is reached;
A fifth step of generating a flicker-corrected captured image by averaging the plurality of captured images obtained in the fourth step;
A calibration method characterized by sequentially performing. A calibration method for performing flicker correction based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection unit,
A first step of determining a luminance level difference between two captured images obtained when the image is continuously captured at the same shutter speed, and determining whether the luminance level difference is a set value or more;
A second step of setting a lower shutter speed that is slower than the same shutter speed while the brightness level difference is equal to or greater than the set value;
A third step of repeating the first step and the second step using the low-speed shutter speed until the lower limit shutter speed is reached;
A fourth step of setting the number of shootings to a plurality of times so that the number of times increases as the luminance level difference increases, according to the luminance level difference when the shutter speed is reached;
A fifth step of continuously shooting the image a plurality of times at the same shutter speed using the lower limit shutter speed;
A sixth step of generating a picked-up image subjected to flicker correction by averaging the plurality of shot images obtained in the fifth step;
A calibration method characterized by sequentially performing. A calibration device that performs flicker correction based on a photographed image obtained by photographing the image in order to adjust the display state of the image projected on the display surface by the projection unit,
Photographing means for photographing the image and outputting the photographed image;
Exposure control means for commanding the shutter speed and aperture of the photographing means;
Brightness level comparison means for determining whether or not the brightness level difference between two captured images obtained when continuously shooting at the same shutter speed by the shooting means is greater than or equal to a set value;
If the brightness level difference does not become less than the set value when the continuous shooting is repeated while gradually reducing the shutter speed of the shooting unit, a shooting number setting unit that sets the number of shootings to a plurality of times;
Correction for generating a picked-up image with flicker correction by averaging the plurality of captured images obtained when the image is continuously captured a plurality of times by the photographing means at a lower limit shutter speed commanded by the exposure control means. Data generation means;
A calibration apparatus comprising:   4. The calibration apparatus according to claim 3, wherein the photographing number setting unit sets the plurality of times so that the number of times increases as the luminance level difference increases.   4. The calibration apparatus according to claim 3, further comprising a sensor for detecting a flicker period of a light source of the projection unit.

Images