JP5110911B2 - Image projection system and image projection method - Google Patents

Description

  The present invention relates to an image projection system and an image projection method for adjusting a positional shift that occurs between a plurality of projection images.

  In recent years, high-definition video processing technology with a sense of reality has rapidly spread, and display of ultra-high-definition video exceeding high-vision is being realized. At present, projection displays are promising for high-definition image display, but the display elements provided in the display have a fine pixel structure, and the positional deviation (pixels) between projected images projected from a plurality of display elements It is necessary to have a technique for automatically adjusting the deviation) with high accuracy.

  Conventionally, since the position of the projected image is generally adjusted by the adjuster through visual observation, the position and the feeling of each adjuster are large, and the adjustment takes a lot of time. For this reason, quantitative evaluation of adjustment and automatic adjustment are required.

As a quantitative position adjustment technique for a projected image, Patent Document 1 describes that X- and Y-direction line sensors are arranged on a screen and adjustment is performed based on signals input from the respective line sensors. Yes. Japanese Patent Application Laid-Open No. H10-228561 describes that the position adjustment between an area that can be seen on the screen and an area that cannot be seen is performed. Non-Patent Document 1 describes, as a method for adjusting the position of a high-definition projected image, a convergence correction by projecting a mountain-shaped image pattern and a method for detecting a positional relationship between display elements by projecting a periodic structure pattern. Yes.
JP-A-8-201937 JP-A-7-123427 Yuichi Kusakabe and two others, "Automation of convergence error and element position adjustment of ultra-high-definition video display system", Journal of the Institute of Image Information and Television Engineers, The Institute of Image Information and Television Engineers, Vol.60, No.2, 2006, pp. 234-241

  In conventional projection image position adjustment, long-time heat running is required before image projection, and position adjustment is performed using the entire screen, so that position adjustment can be performed before the start of projection of a desired image. Done. However, after the position adjustment, during the projection of the desired image, the misalignment of the projected image caused by heat, etc. is automatically corrected and cannot always be optimally adjusted, and stable high-definition image display It was difficult.

  The present invention has been made in view of the above-described problem, and an object thereof is to provide an image projection system and an image projection method capable of adjusting the position of a projection image even during projection of a desired projection image. To do.

The present invention has been made to solve the above-described problems, and projects a projection image on a first display area and a pattern image for detecting misalignment in a second display area different from the first display area. A plurality of image projecting means for projecting the first image projecting means for projecting the first pattern image whose brightness periodically changes at a predetermined spatial frequency, and the brightness periodically at the predetermined spatial frequency. Second image projecting means for projecting a second pattern image that changes to the first pattern image and having a spatial phase characteristic different from that of the first pattern image while temporally changing a phase amount given to the second pattern image. comprising, a pattern image detection means for detecting a composite image of the projected first pattern image and the second pattern image on the second display area, the pattern image detected by the detecting means a first Based on the aliasing components occurring in the composite image of the pattern image second pattern image, and positional deviation amount detecting means for detecting a positional deviation amount between the projection image projected from the plurality of image projection means, said position An image projection system comprising: a position adjustment unit that adjusts a position of the projection image based on the positional deviation amount detected by the deviation amount detection unit.

  In addition, the image projection system of the present invention includes a first image projection unit and a second image projection unit as the plurality of image projection units, and the first image projection unit has a periodicity with a predetermined spatial frequency. A first pattern image that changes periodically, and the second image projecting means is a second that has a spatial phase characteristic different from that of the first pattern image because the luminance periodically changes at the predetermined spatial frequency. The positional deviation amount detection unit is configured to detect the positional deviation based on a aliasing component generated in a composite image of the first pattern image and the second pattern image detected by the pattern image detection unit. It is characterized by detecting the quantity.

  In the image projection system of the present invention, the spatial phase characteristic of the second pattern image changes along a direction in which the spatial phase characteristic of the first pattern image is constant.

  In the image projection system of the present invention, the pattern image detection means is arranged in the second display area.

Further, the present invention is a pattern image for detecting misalignment in a second display area different from the first display area while projecting a projection image from a plurality of image projecting means to the first display area , A first pattern image whose luminance periodically changes at a predetermined spatial frequency, and a second pattern image whose luminance periodically changes at the predetermined spatial frequency and which has a different spatial phase characteristic from the first pattern image Are projected while temporally changing the phase amount given to the second pattern image, and a composite image of the first pattern image and the second pattern image projected on the second display area Detecting the amount of positional deviation between the projected images projected from the plurality of image projecting means based on the detected aliasing component generated in the synthesized image of the first pattern image and the second pattern image Shi An image projection method characterized by adjusting the position of the projection image based on the detected positional deviation amount.

  According to the present invention, by projecting a projection image on the first display area, projecting a pattern image for detecting displacement in the second display area, and adjusting the position of the projection image based on the pattern image. The effect that the position of the projection image can be adjusted even during the projection of the desired projection image is obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a state of image projection in one embodiment of the present invention. An image is projected onto the screen 2 (first display area) by the image projection device 1, and the observer 3 observes the display image 4 displayed on the screen 2. At the same time, a pattern image 6 for detecting displacement is projected onto the screen frame 5 (second display area) by the image projection apparatus 1. This pattern image 6 is captured by the camera 7. Since the observer 3 observes the display image 4 from the side opposite to the side on which the pattern image 6 is projected, the pattern image 6 is not visible to the observer 3, and the pattern image 6 is observed even while the display image 4 is being projected. It won't get in the way.

  FIG. 2 shows the configuration of the image projection system according to the present embodiment. The projection image is reproduced from the display image signal source 8, and the pattern image for detecting the displacement is generated from the detection pattern generator 9. These are synthesized by an image synthesizer 10 such as a bidet switcher and input to the image signal processing device 11. The total number of pixels of the pattern image may be smaller than that of the projected image, and the same pattern need only be displayed in a plurality of areas of the screen frame 5. In addition, an image obtained by synthesizing the detection pattern with the projection image may be reproduced from the display image signal source 8.

  The image signal processing device 11 corrects the distortion or misalignment of the projected image. An output signal of the image signal processing device 11 is input to the image projection device 1. The image projection apparatus 1 has a plurality of display elements. In this embodiment, as an example, the display element 13r corresponding to the R signal, the display elements 13g1, 13g2, corresponding to the two types of G signals, and the B signal are supported. A display element 13b is provided. An image based on each signal is incident on these display elements, optically synthesized, and projected onto the screen 2 and the screen frame 5.

  As these display elements, transmissive or reflective liquid crystal devices, digital micromirror devices, grating light valves, and the like can be used. The position of these display elements can be finely adjusted by the electrically controlled fine movement stage 14. As the fine movement stage 14, a stepping motor, a piezoelectric element or the like can be used, and it is desirable to use a stage having a resolution of about 1/50 or less of the size of one pixel of the display element. .

  The pattern image 6 projected on the screen frame 5 is picked up by the camera 7, and an image signal is output from the camera 7 to the misalignment calculation device 15. The camera 7 has a light receiving sensor such as an arrayed CCD or CMOS sensor or a photodiode, and a pattern image incident through the camera lens is formed at the sensor position. The misregistration calculation device 15 calculates (detects) the misregistration amount between the images projected by the display elements 13r, 13g1, 13g2, and 13b based on the image signal from the camera 7. The misregistration calculation device 15 can also calculate the amount of image distortion.

  A signal based on the calculated misregistration amount is output from the misregistration calculation device 15 to the image signal processing device 11 or the stage control device 16. The image signal processing device 11 corrects (adjusts) an image signal for forming an image to be incident on each display element by image processing based on a signal from the position shift calculation device 15 so that the position shift is eliminated. Further, the stage control device 16 drives the fine movement stage 14 to perform position control in the horizontal direction and the vertical direction and, if necessary, the rotational direction so as to eliminate the positional deviation, and thereby display elements 13r, 13g1, 13g2, and 13b. Adjust the position. It is also possible to switch and display a plurality of pattern images for position shift detection based on a control signal output from the position shift calculation device 15 to the detection pattern generation device 9.

  Next, a method for detecting misalignment in the present embodiment will be described. As a method for displaying high-definition video, there is a method of using two display elements for green images and synthesizing images from each display element by shifting by half a pixel, which is an effective method for future high-definition video display. (M. Kanazawa, et al., Journal of SID, Vol. 12, No. 1, 2004, pp. 93-103). This method can also be applied to this embodiment.

  In image display that is rapidly becoming higher in definition, more accurate positional deviation detection and adjustment techniques are required. However, for the position adjustment of two green images, particularly high-precision position detection / correction is required. Technology is required. As an effective method for detecting misalignment between a plurality of images, there is a method of projecting a pattern having a luminance distribution of a periodic structure and detecting an aliasing component of an image caused by misalignment between the images (described above). Non-patent document 1). Hereinafter, an example in which this method is applied to the present embodiment will be described.

  In the present embodiment, the first pattern image 300 and the second pattern image 310 shown in FIG. 3 are used in order to detect a horizontal displacement between two images. The first pattern image 300 has a periodic luminance distribution in which the luminance changes at a predetermined spatial frequency in the horizontal direction. The second pattern image 310 also has a periodic luminance distribution in which the luminance changes at the same spatial frequency as the first pattern image 300, but has a spatial phase characteristic different from that of the first pattern image 300. The first pattern image 300 and the second pattern image 310 have a phase difference 320.

  In FIG. 3, the pattern portion 300a is a portion with high luminance, and the pattern portion 300b is a portion with low luminance. In FIG. 3, the change in luminance in each of the pattern portions 300a and 300b is not shown due to restrictions on the representation of the figure, but actually the luminance changes smoothly along the horizontal direction. And

  The first pattern image 300 is incident on a reference display element (hereinafter referred to as a first display element), and the second pattern image 310 is a display element (hereinafter referred to as a first display element) that projects an image to be aligned. 2 display element). Further, the first pattern image 300 and the second pattern image 310 are projected on the same position of the screen frame 5. As will be described below, the misalignment calculation device 15 detects the aliasing component caused by the misalignment in the composite image of the two pattern images while changing the phase difference between the two pattern images in a time division manner. A minimum phase difference is obtained, and a positional deviation amount from the normal position is detected based on the phase difference.

As an example, a method for detecting a displacement in the horizontal direction (x direction) will be described. A first pattern image whose luminance distribution in the x direction is represented by g ₁ in the following equation (1) is displayed (projected) on the first display element. Here, f is a spatial frequency of the pattern, i (i = 1, 2,..., N) is a pixel position, and δ is a delta function.

In addition, a position that is shifted by a half pixel from the first pattern image with respect to the second display element is a normal position, and a luminance distribution in the x direction is a _second expressed by g ₂ in the following equation (2). The pattern image is displayed (projected).

  If the position of the second pattern image displayed by the second display element is shifted by q pixels, and the phase difference added to the second pattern image with the first pattern image as a phase reference is 2πfp, The condition that the aliasing component does not occur is expressed by the following equation (3). That is, by obtaining the phase p that minimizes the aliasing component, the positional deviation amount q from the normal position can be obtained. In addition, as a periodic structure of the luminance distribution of the first pattern image and the second pattern image, a rectangular wave distribution may be used in addition to the sine wave distribution as described above.

  Hereinafter, a specific procedure for detecting misregistration will be described with reference to FIG. The detection pattern generation device 9 generates a first pattern image which is a fixed pattern and also generates a second pattern image having a predetermined phase difference with respect to the first pattern image (step S100). After the two pattern images are separately synthesized with the projection image by the image synthesizer 10 and processed by the image signal processing device 11, the first pattern image is projected by the first display element, and the second pattern image is obtained. Is projected by the second display element (step S110).

  A composite image of the first pattern image and the second pattern image displayed on the screen frame 5 is picked up by the camera 7, and an image signal is output to the misalignment calculation device 15 (step S120). Based on the image signal from the camera 7, the position shift calculation device 15 extracts an image of a region on which a position shift detection pattern image is projected from the captured image (step S 130).

  The position shift calculation device 15 performs FFT (Fast Fourier Transform) on the image in this region (step S140), and detects the magnitude of the aliasing component (step S150). When there is a condition in which the aliasing component has not yet been detected among the preset phase difference conditions (NO in step S160), the detection pattern generator 9 performs control output from the misalignment calculator 15. Based on the signal, the phase amount to be given to the second pattern image is changed (step S170).

  Steps S100 to S170 are repeated while slightly changing the phase amount of the second pattern image until the aliasing component is detected under all the preset phase difference conditions. When the aliasing component is detected under all the preset phase difference setting conditions, the misalignment calculation device 15 obtains the minimum value of the aliasing component detected under each phase difference setting condition (step S180). Further, the misregistration calculation device 15 calculates the misregistration amounts of the two pattern images from the phase amount corresponding to the minimum value by the equation (3), and the misregistration amounts obtained at a plurality of locations on the screen frame 5. The amount of displacement on the screen 2 is calculated by performing an interpolation operation (step S190).

  In order to accurately detect the amount of positional deviation by the above method, it is desirable to reduce the amount of change in the phase amount given to the second pattern image and increase the number of repetitions of steps S100 to S170. Alternatively, the accuracy may be increased by increasing the amount of change in the phase amount and roughly detecting the amount of positional deviation, and then detecting the amount of positional deviation by reducing the amount of change in the phase amount. In the above method, in the case of detecting a positional deviation in the horizontal direction, the images may be averaged in the vertical direction of the image to reduce random noise superimposed on the image.

  Next, another method for detecting misalignment will be described. The first pattern image 500 shown in FIG. 5 may be displayed on the first display element, and the second pattern image 510 may be displayed on the second display element. Although the spatial frequency characteristics of the first pattern image 500 and the second pattern image 510 are the same, the spatial phase characteristics of the second pattern image 510 are directions in which the spatial phase characteristics of the first pattern image 500 are constant ( It changes along the vertical direction of FIG. That is, the phase difference at the same vertical position of the first pattern image 500 and the second pattern image 510 changes according to the vertical position.

  Hereinafter, the procedure for detecting misalignment using the two pattern images shown in FIG. 5 will be described with reference to FIG. The detection pattern generator 9 generates the first pattern image and the second pattern image shown in FIG. 5 (step S200). After the two pattern images are separately synthesized with the projection image by the image synthesizer 10 and processed by the image signal processing device 11, the first pattern image is projected by the first display element, and the second pattern image is obtained. Is projected by the second display element (step S210).

  A composite image of the first pattern image and the second pattern image displayed on the screen frame 5 is picked up by the camera 7, and an image signal is output to the misalignment calculation device 15 (step S220). Based on the image signal from the camera 7, the position shift calculation device 15 extracts an image of an area where a pattern image for position shift detection is projected from the captured image (step S 230).

  The positional deviation calculation device 15 performs FFT on the image in this region (step S240), and detects the magnitude of the aliasing component (step S250). At this time, the positional deviation calculation device 15 detects the magnitude of the folding component at a plurality of vertical positions in FIG.

  Subsequently, the positional deviation calculation device 15 obtains the minimum value of the aliasing component detected at each vertical position (step S260). Further, the misregistration calculation device 15 calculates the misregistration amounts of the two pattern images from the phase amount corresponding to the minimum value by the equation (3), and the misregistration amounts obtained at a plurality of locations on the screen frame 5. The amount of displacement on the screen 2 is calculated by performing an interpolation operation (step S270). As described above, by using the first pattern image and the second pattern image having different phase differences at each position in the vertical direction, the repetitive operation generated by the method of changing the phase difference in the time division manner shown in FIG. Is unnecessary, and position adjustment can be performed at high speed.

  In the above description, the method for detecting the positional deviation in the horizontal direction has been described, but the method for detecting the positional deviation in the vertical direction is the same. For example, as shown in FIG. 7, a first pattern image 700 having a periodic luminance change in the vertical direction is displayed on the first display element serving as a reference, and the spatial phase characteristics at each position in the horizontal direction are displayed. The changing second pattern image 710 may be displayed on the second display element that projects the image to be aligned, and the positional deviation may be detected by the same method as described above.

  Further, in order to detect the positional deviation in the vertical direction and the horizontal direction within one screen, the following may be performed. That is, as shown in FIG. 8, the first pattern image 500 and the second pattern image 510 for detecting the positional deviation in the horizontal direction in FIG. 5 are projected onto the first area 800 in the screen frame 5, and FIG. The first pattern image 700 and the second pattern image 710 for detecting the positional deviation in the vertical direction are projected onto the second region 810 in the screen frame 5.

  The misregistration calculation device 15 calculates the horizontal misregistration amount from the image of the first region 800 among the images captured by the camera 7 and calculates the misalignment amount of the vertical direction from the image of the second region 810. To do. Based on the calculated amount of displacement, the position of the projected image in the horizontal and vertical directions is adjusted.

  Alternatively, the following may be used. That is, the first state in which the first pattern image 500 and the second pattern image 510 for detecting the displacement in the horizontal direction in FIG. 5 are projected on the screen frame 5, and the detection in the vertical direction in FIG. Switching between the second state in which the first pattern image 700 and the second pattern image 710 are projected onto the screen frame 5 is performed in a time-sharing manner.

  For this reason, the displacement calculation device 15 outputs a control signal to the detection pattern generation device 9 and switches the first state and the second state, and the amount of displacement in the horizontal direction from the image captured in the first state. And the amount of vertical displacement is calculated from the image captured in the second state. Based on the calculated amount of displacement, the position of the projected image in the horizontal and vertical directions is adjusted.

  According to this method, it is possible to detect a positional shift in each direction by using a larger area of the screen frame 5 than a method of simultaneously projecting a pattern image for detecting a positional shift in the horizontal direction and the vertical direction. As a result, it is possible to improve the accuracy of the positional deviation detection. Further, when the display area of the pattern image is limited to a relatively narrow area, the frequency f of the brightness change of the pattern image is optimally selected, and the aliasing component is generated in a frequency band suitable for the display area. It may be.

  Next, another example of the arrangement position of the light receiving sensor will be described. In the above description, the pattern image is captured by the camera 7 having the light receiving sensor. However, when the size of the projection image is relatively small, the light receiving sensor may be directly mounted on the screen frame 5. FIG. 9 shows a state in which the light receiving sensor 70 is arranged in the screen frame 5.

  When the light receiving sensor 70 is arranged in the screen frame 5, the pattern image projected from the image projection device 1 is directly incident on the light receiving sensor 70. As a result, the camera lens is not affected by distortion or vignetting as compared with the case where the camera 7 captures a pattern image. Further, since the pattern image is directly formed on the light receiving sensor 70, the light can be received with high definition and high efficiency. Furthermore, the influence of the brightness and reflection of the projected image displayed on the screen 2 can be reduced, which is advantageous for detecting misalignment during image projection.

  As described above, according to the present embodiment, an image is projected onto the screen 2, a pattern image for detecting misalignment is projected onto the screen frame 5, and misalignment between pattern images is detected based on the pattern image. Adjust the position of the projected image. As a result, the position of the projection image can be adjusted even during projection of a desired image, and an optimum projection image can be displayed at all times. In addition, the configuration relating to the position shift detection can be made relatively simple.

  In addition, by projecting two types of pattern images having the same spatial frequency and different spatial phase characteristics, and detecting the amount of misalignment based on the aliasing component generated in the composite image, the position can be accurately detected with less than half a pixel. The amount of deviation can be detected with high accuracy.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

It is a reference figure which shows the mode of the image projection in one Embodiment of this invention. It is a block diagram which shows the structure of the image projection system by one Embodiment of this invention. It is a reference figure showing typically a pattern picture in one embodiment of the present invention. It is a flowchart which shows the procedure of the position shift detection in one Embodiment of this invention. It is a reference figure showing typically a pattern picture in one embodiment of the present invention. It is a flowchart which shows the procedure of the position shift detection in one Embodiment of this invention. It is a reference figure showing typically a pattern picture in one embodiment of the present invention. It is a reference diagram for explaining a detection method of a positional deviation in the horizontal direction and the vertical direction in an embodiment of the present invention. In one Embodiment of this invention, it is a reference figure which shows the state which has arrange | positioned the light receiving sensor in the screen frame.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image projection apparatus, 2 ... Screen, 5 ... Screen frame, 7 ... Camera (pattern image detection means), 8 ... Display image signal source, 9 ... Detection pattern generator DESCRIPTION OF SYMBOLS 10 ... Image synthesizer, 11 ... Image signal processing apparatus (position adjustment means), 13r, 13g1, 13g2, 13b ... Display element (image projection means), 14 ... Fine movement stage, 15. ..Position displacement calculation device (position displacement amount detection means, state switching means), 16... Stage control device (position adjustment means), 70... Light receiving sensor (pattern image detection means)

Claims (4)

A plurality of image projecting means for projecting a projection image onto the first display area and projecting a pattern image for detecting misregistration onto a second display area different from the first display area , at a predetermined spatial frequency; A first image projecting means for projecting a first pattern image whose luminance changes periodically, and a first phase image having a spatial phase characteristic different from that of the first pattern image whose luminance periodically changes at the predetermined spatial frequency. A second image projecting means for projecting the pattern image of 2 while changing the phase amount given to the second pattern image temporally ,
Pattern image detection means for detecting a composite image of the first pattern image and the second pattern image projected on the second display area;
A positional deviation between the projected images projected from the plurality of image projecting units based on a aliasing component generated in a composite image of the first pattern image and the second pattern image detected by the pattern image detecting unit. A positional deviation amount detecting means for detecting the amount;
Position adjustment means for adjusting the position of the projection image based on the position deviation amount detected by the position deviation amount detection means;
An image projection system comprising: The image projection system according to claim 1 , wherein the spatial phase characteristic of the second pattern image changes along a direction in which the spatial phase characteristic of the first pattern image is constant. Image projection system of claim 1 or claim 2, wherein the pattern image detecting means is disposed in said second display area. A projection image is projected from a plurality of image projection means to the first display area, and is a pattern image for detecting misalignment in a second display area different from the first display area, and the luminance is at a predetermined spatial frequency. A first pattern image that periodically changes, and a second pattern image whose luminance periodically changes at the predetermined spatial frequency and having a spatial phase characteristic different from that of the first pattern image. Project while changing the phase amount to be given to the pattern image of time ,
Detecting a composite image of the first pattern image and the second pattern image projected on the second display area;
Based on the aliasing component generated in the composite image of the detected first pattern image and the second pattern image, a displacement amount between the projection images projected from the plurality of image projection means is detected,
An image projection method, wherein the position of the projection image is adjusted based on the detected displacement amount.

Images