JP2011228832A - Image processing device, image display system, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device for creating image data to be input into an image display system that displays a pattern for position correction to correct a positional deviation.SOLUTION: An image processing device creates a first projection image data showing a first projection image PA and a second projection image data showing a second projection image PB. The first projection image PA is a first image P1 with a first pattern TP1 added for positioning and the second projection image PB is a second image P2 with a second pattern TP2 corresponding to the first pattern TP1 added. Based on a first image data showing the first image P1 and a second image data showing the second image P2, the image processing device creates the first projection image data and the second projection image data by making the luminosities of the first pattern TP1 and the second pattern TP2 different from each other complementarily so that the sum luminosity of the first pattern TP1 luminosity and the second pattern TP2 luminosity is equal to the luminosity of a corresponding part in a given image.

Description

  The present invention relates to an image processing apparatus, an image display system, and an image processing method.

  Conventionally, projectors are used for presentations such as exhibitions, academic conferences, and conferences, or home theaters. In recent years, the use of projectors has expanded to store advertisements. When a projector is used, it is easier to display a large screen than a direct-view display device such as a display. Therefore, the same image can be shown to a larger number of people, and convenience is high. In recent years, various developments have been made in response to market demands for high-quality projected images.

  For example, the resolution of a projected image by a projector depends on the number of pixels of an image forming element such as a liquid crystal light valve or DMD (digital mirror device) that the projector has. Therefore, in order to increase the resolution of the projected image projected by the projector, it is necessary to use a high-resolution image forming element. However, such a high-definition image forming element tends to be very expensive and is not suitable for general use.

  Therefore, a system for projecting one image using a plurality of projectors has been developed. Patent Literature 1 describes an image display system capable of displaying one high-definition image by superimposing images projected from a plurality of projectors on a projection surface. Moreover, in the image display system described in Patent Document 1, a plurality of projectors project and display inspection patterns for detecting the relative positions of the projectors, and the positions of the projectors detected from the relative positions of the inspection patterns. The deviation is corrected mechanically.

JP-A-8-168039

  By the way, it is conceivable that the relative positions of images projected from a plurality of projectors change with time due to drive heat generation inside the projector, thermal deformation of the optical member due to temperature changes in the use environment, and the like. Therefore, it is desirable to readjust the relative position of the image projection position even when the image display system is in use.

  However, in the image display system of Patent Document 1, an inspection pattern is projected and displayed from each projector, the inspection pattern is photographed with a camera, a relative position is measured, and then a position correction amount is calculated based on the measured relative position. It is necessary to follow the procedure of calculating and correcting the position. Therefore, when correcting the positional deviation of the projection position while using the image display system, it is necessary to project and display the inspection pattern after erasing the projected image. If an attempt is made to correct misalignment while displaying an image, the inspection pattern is superimposed on the projected image, and thus the inspection pattern is recognized even by a person viewing the projected image, resulting in poor usability.

  The present invention has been made in view of such circumstances, and provides an image display system capable of displaying a pattern for position correction without causing the user to recognize the position and correcting the position shift. With the goal. It is another object of the present invention to provide an image processing apparatus and an image processing method for creating projection image data indicating a projection image projected from the image display system.

  In order to solve the above-described problem, the image processing apparatus of the present invention is configured to input the first projection input to an image display system that displays a predetermined image by projecting the first projection image and the second projection image in a superimposed manner. An image processing apparatus for creating first projection image data indicating an image and second projection image data indicating the second projection image, wherein the first projection image is obtained by adding a first pattern for positioning to the first image. The second projected image is an image obtained by adding a second pattern corresponding to the first pattern to the second image, the first image data indicating the first image, and the second image indicating the second image. The first pattern brightness and the second pattern brightness based on two image data so that the brightness is the same as the brightness of the corresponding portion in the predetermined image. Patterns and Varied complementary brightness of the second pattern, characterized by creating a first projection image data and the second projection image data.

  According to this configuration, the image data of two projection images in which the brightness of the first pattern and the second pattern are complementarily different are created, so that the first and first patterns can be displayed to the user when displayed on the image display system. It is possible to display a projection image in which the two patterns are not recognized or the first and second patterns are not conspicuous.

In the present invention, as the first projection image, an image obtained by adding the first pattern to the first image and an image obtained by adding the first pattern to the second image are obtained, and are each paired. The first pattern included in the image obtained by obtaining the second projection image and adding the first pattern to the first image and the image obtained by adding the first pattern to the second image uses green light. It is desirable to be prescribed.
Since the human eye has the highest visual sensitivity, green is preferable when position correction is performed on the basis of the first pattern formed with green color light because the positional deviation of each projected image is difficult to be recognized by the human eye.

In the present invention, the first projection image and the second projection image are projected on the projection surface while being shifted from each other by a half pixel, and the first projection image is the first image. To obtain an image with the first pattern and an image with the first pattern added to the second image, and to obtain the second projection image to be paired with each other, and attach the first image to the first image. The first pattern may be defined using one of red light and blue light, and the first pattern attached to the second image may be defined using the other of red light and blue light. desirable.
When the pixels of the first projection image and the second projection image are superimposed on each other by shifting 0.5 pixels in the vertical and horizontal directions, the resolution of a predetermined image displayed in cooperation with the first projection image and the second projection image Can be high. In this case, when the positioning pattern is displayed with green color light, the resolution is lowered at the portion where the first pattern and the second pattern are overlapped. In such a case, by displaying the positioning pattern with red and blue color light, the image formed with the green color light maintains high resolution, and the position of other colors can be corrected. Therefore, it is possible to create projection image data capable of displaying a higher quality image.

In the present invention, it is desirable that the first pattern has a shape in which two or more coordinates are specified.
According to this configuration, the relative position between the first projected image and the second projected image can be detected based on the coordinates of the two feature points. Then, when the first projected image and the second projected image are tilted within the projection surface, the tilt between the projected images can be detected from the coordinates of the two feature points. Therefore, more accurate position correction can be performed than position correction based on one feature point.

  Moreover, the image display system of the present invention uses the s-polarized light based on the above-described image processing apparatus and the projection image data supplied from the image processing apparatus, and either the first projection image or the second projection image. Based on the first image forming optical system for forming one of the images and the projection image data supplied from the image processing device, the other image of the first projection image or the second projection image using p-polarized light A second image forming optical system for forming the image, a projection optical system for displaying the predetermined image by superimposing the first projection image and the second projection image on a projection surface, and s-polarized light or p-polarized light Any one of the first projection image and the second projection image projected on the projection surface when observing the predetermined image. Selectively project the projected image Projection position of the first projection image and the second projection image on the projection surface obtained from the imaging unit and the captured images of the first projection image and the second projection image captured by the imaging unit The position of at least one of the first projection image and the second projection image so that the relative position between the first projection image and the second projection image approaches a predetermined target position based on And a position correcting means for moving.

  According to this configuration, it is possible to provide an image display system capable of displaying a positioning pattern between the first projection image and the second projection image without causing the user to recognize the position and correcting the positional deviation. It becomes. Therefore, when the relative position between the first projection image and the second projection image changes with time due to drive heat generation inside the projector, thermal deformation of the optical member due to temperature change in the use environment, etc., the image display system Even during use, it is possible to readjust the relative position of the image projection position.

In the present invention, it is desirable that the imaging unit includes a first imaging unit having a polarizing element that transmits s-polarized light and a second imaging unit having a polarizing element that transmits p-polarized light.
According to this configuration, since the first pattern included in the first projection image and the second projection image can be simultaneously captured by the two imaging units, the imaging timing such as the measurement error due to the vibration of the projector can be reduced. An error caused by the difference does not occur, and the accuracy of position correction can be increased.

In the present invention, the projection optical that projects the first image forming optical system, the second image forming optical system, the first projection image, and the second projection image so as to be superimposed on the projection surface. And a projector comprising the system.
By using a so-called six-plate type projector, the apparatus configuration can be simplified, the positional deviation is corrected, and the first projected image and the second projected image are reliably superimposed and displayed. It can be.

  Further, the image processing method of the present invention provides a first projection indicating the first projection image that is input to an image display system that displays a predetermined image by superimposing the first projection image and the second projection image. An image processing method for creating image data and second projection image data indicating the second projection image, wherein the first projection image is an image obtained by adding a first pattern for positioning to the first image, and The two projection images are images obtained by adding a second pattern corresponding to the first pattern to the second image, and include first image data indicating the first image and second image data indicating the second image. Based on the first pattern and the second pattern, the brightness obtained by adding the brightness of the first pattern and the brightness of the second pattern is the same as the brightness of the corresponding portion in the predetermined image. of Degrees complementary to at different, characterized by creating a first projection image data and the second projection image data.

  According to this method, image data of two projection images in which the brightness of the first pattern and the second pattern are complementarily different are created, so that the first and first patterns can be displayed to the user when displayed on the image display system. It is possible to display a projection image in which the two patterns are not recognized or the first and second patterns are not conspicuous.

1 is a schematic diagram illustrating an overall configuration of an image display system according to a first embodiment. It is a schematic sectional drawing which shows the image forming element of a projector. It is a figure explaining the projection image displayed on a to-be-projected surface. It is a figure which shows the example of the projection image projected and displayed by an image display system. It is a figure explaining a test pattern. It is explanatory drawing which shows a 1st projection image and a 2nd projection image. It is a figure explaining the modification of a test pattern. It is a figure explaining the modification of a test pattern. It is explanatory drawing of the image display system which concerns on 2nd Embodiment. It is explanatory drawing of the projector which the image display system of 3rd Embodiment has. It is explanatory drawing of the projector which the image display system of 3rd Embodiment has.

[First Embodiment]
Hereinafter, an image processing apparatus, an image display system, and an image processing method according to an embodiment of the present invention will be described with reference to FIGS. In all the drawings below, the dimensions and ratios of the constituent elements are appropriately changed in order to make the drawings easy to see.

[Device configuration]
1 and 2 are schematic views showing an image display system according to the present embodiment, FIG. 1 is a schematic view showing the overall configuration of the image display system 1A, and FIG. 2 shows an image forming element of a projector provided in the image display system 1A. It is a schematic sectional drawing shown.

  As shown in FIG. 1, the image display system 1 </ b> A of this embodiment includes a plurality of projectors 2, an image processing device 3, and a digital camera (imaging unit) 4. Here, a state is shown in which the user X is looking at the projection image P displayed using the image display system 1A.

  Although two projectors 2 are arranged in FIG. 1, the number and arrangement are not limited to this. Here, it is assumed that a first projector 21 and a second projector 22 are provided as the projector 2. Each projector 21, 22 has an image forming element (image forming optical system) for forming a projected image. As the image forming element, a liquid crystal light valve, DMD (digital mirror device), or the like can be used. Here, it is assumed that the projector 2 has a liquid crystal light valve as an image forming element.

  For example, as shown in FIG. 2, the liquid crystal light valve has a liquid crystal layer 204 sealed in a space surrounded by a pair of substrates 201 and 202 and a sealing material 203. The pair of substrates 201 and 202 includes a pixel electrode (not shown), a common electrode, an alignment film, and the like that sandwich the liquid crystal layer 204. Further, on the opposite side of the pair of substrates 201 and 202 from the liquid crystal layer 204, polarizing elements 205 and 206 are provided in a crossed Nicols arrangement or a parallel Nicols arrangement.

  The first projector 21 and the second projector 22 are designed so as to form a projected image by emitting linearly polarized light whose vibration directions are orthogonal to each other. That is, the liquid crystal light valve of the first projector 21 is provided with a polarizing element so that the transmitted light (image light) is s-polarized light, and the liquid crystal light valve of the second projector 22 is transmitted light ( A polarizing element is provided so that (image light) becomes p-polarized light. Of course, the first projector 21 may emit p-polarized light and the second projector 22 may emit s-polarized light.

  The first projector 21 displays the first projection image PA on the projection surface S based on the image data input from the image processing device 3. Further, the second projector 22 displays the second projected image PB on the projection surface S based on the image data input from the image processing device 3.

  Since the image light from the liquid crystal light valve of the first projector 21 is s-polarized light, the image light that forms the first projection image PA is s-polarized light. On the other hand, since the image light from the liquid crystal light valve of the second projector 22 is p-polarized light, the image light that forms the second projection image PB is p-polarized light. The user X recognizes the image displayed on the projection surface S as one projection image P in which the first projection image PA and the second projection image PB overlap.

  Further, the projection lens 29 of the second projector 22 is provided with a projection position control mechanism for enabling the projection position of the second projection image PB to move in the plane direction on the projection surface S. The projection position control mechanism moves the projection position of the second projection image PB according to an instruction from the image processing device 3.

  The image processing apparatus 3 creates image data corresponding to the images PA and PB projected and displayed by each projector from the image data input from the signal source 9. The first and second projectors 21 and 22 display images on the projection surface S based on the image data input from the image processing device 3.

  That is, the image processing device 3 outputs image data indicating the first projection image PA to the first projector 21. Similarly, the image processing device 3 outputs image data indicating the second projection image PB to the second projector 22. The image light that forms the first projection image PA is s-polarized light, and the image light that forms the second projection image PB is p-polarized light.

  The digital camera 4 is a device that captures a projected image P projected and displayed on the projection surface S. The digital camera 4 includes a lens, a photoelectric conversion element, a polarization element, and the like. As the photoelectric conversion element, a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, or the like is used. The polarizing element is provided in the optical path until light incident through the lens reaches the photoelectric conversion element, and selectively transmits either s-polarized light or p-polarized light. The polarizing element is configured so that the linearly polarized light to be transmitted can be arbitrarily changed between s-polarized light and p-polarized light, and the first projected image PA depends on whether the polarizing element transmits s-polarized light or p-polarized light. And only one of the second projected image PB can be captured.

[Projected image]
Next, the projection image P projected and displayed by the image display system 1A of the present embodiment will be described with reference to FIGS.

  3A and 3B are diagrams for explaining the projection image P displayed on the projection surface S. FIG. 3A is a diagram showing the first projection image PA or the second projection image PB, and FIG. It is a figure which shows the projection image P displayed by superimposing and displaying the 1st projection image PA and the 2nd projection image PB.

  As shown in FIG. 3A, the first projection image PA and the second projection image PB are images having a resolution of m × n pixels in which pixels Px are arranged in m pixels in the x direction and n pixels are arranged in the y direction. The distance between the pixels Px is equal to the width of the pixel Px.

  As shown in FIG. 3B, in the image display system 1A of the present embodiment, the first projection image PA and the second projection image PB are displayed with a shift of 0.5 pixels in the horizontal direction and the vertical direction, respectively. Thereby, the projection image P displayed by the first projection image PA and the second projection image PB is an image having twice the number of pixels as the first projection image PA and the second projection image PB.

  FIG. 4 is a diagram illustrating an example of the projected image P. In the first projection image PA and the second projection image PB constituting such a projection image P, a first test pattern (first pattern) TP1 and a second test shown in FIG. A pattern (second pattern) TP2 is provided.

  As shown in FIG. 5A, in the first test pattern TP1, a circular pattern Xa having a high brightness is provided at the center, and a pattern Xb having a lower brightness than the pattern Xa is provided around the pattern Xa. Yes. On the other hand, as shown in FIG. 5B, in the second test pattern TP2, a circular pattern Xc having a low brightness is provided at the center, and a pattern Xd having a higher brightness than the pattern Xc is provided around the pattern Xc. Is provided.

  The circle indicated by the pattern Xa is used to specify coordinates on the projection surface of the first test pattern TP1. That is, the captured image captured by the digital camera 4 is binarized to extract the contour of the figure included in the captured image, and the center coordinates of the circle obtained by expressing the contour by a mathematical expression are the coordinates of the first test pattern TP1. (Coordinates of feature points).

  Further, the pattern Xa included in the first test pattern TP1 is not necessarily circular, and various shapes can be adopted. For example, when the pattern Xa is a cross hatch, the coordinates of the intersection of two lines when the cross hatch is expressed by an equation can be obtained as the coordinates of the first test pattern TP1.

  FIG. 6 is an explanatory diagram showing the first projection image PA and the second projection image PB. FIG. 6A shows the first projection image PA including the first test pattern TP1, and FIG. 6B shows the first projection image PA. It is a figure which shows the 2nd projection image PB containing 2 test pattern TP2.

  As shown in the figure, the first projection image PA includes a first image P1 that constitutes a part of the projection image P, and a first test pattern TP1 provided at a position corresponding to the area AR in the projection image P. Thus, the second projected image PB includes a second image P2 constituting a part of the projected image P and a second test pattern TP2 provided at a position corresponding to the area AR in the projected image P. Each test pattern is drawn by lowering the brightness of the corresponding part in the first image P1 or the second image P2.

  The first test pattern TP1 and the second test pattern TP2 are complementarily different from each other in brightness so that the first test pattern TP1 and the second test pattern TP2 have the same brightness as the area AR in the projection image P. Specifically, the brightness when the pattern Xa and the pattern Xc are overlaid and the brightness when the pattern Xb and the pattern Xd are overlaid correspond to the corresponding portions in the area AR of the projected image P in FIG. It has become the same brightness.

[Image display system functions]
In the image display system 1A of the present embodiment, the first and second projection images PA and PB are obtained by embedding the first test pattern TP1 and the second test pattern TP2 in the original data of the projection image as follows. Correct the position.

[Image processing in image processing apparatus]
First, the image processing apparatus 3 calculates projection image data indicating the first projection image PA including the first test pattern TP1 and the second projection image PB including the second test pattern TP2 as follows.

  In the following description, an image signal input to the pixel of the liquid crystal light valve corresponding to the pixel at the x-th pixel position (x, y) in the x-direction and the y-th pixel position (x, y) with the upper left corner of the image as the origin. Let it be expressed as Image (x, y). Image (x, y) is a combination of the R component image signal ImageR (x, y), the G component image signal ImageG (x, y), and the B component image signal ImageB (x, y).

  The image signal in the first image P1 is represented as Image1 (x, y), and the image signal in the second image P2 is represented as Image2 (x, y). The RGB components are assumed to be Image1R (x, y), Image1G (x, y), Image1B (x, y), Image2R (x, y), Image2G (x, y), and Image2B (x, y), respectively.

  Further, the brightness value is indicated by 0 to 1.0. For example, when black is displayed, all of ImageR, ImageG, and ImageB indicate 0, and when white is displayed, any of ImageR, ImageG, and ImageB is displayed. Shall have a value of 1.0.

  First, image data indicating the first image P1 and the second image P2 that do not have a test pattern is input to the image processing apparatus 3, and the test pattern of the first image P1 and the second image P2 is displayed. It is determined whether a test pattern can be added. Specifically, in the area corresponding to the area AR in the projection image P of the first image P1 and the second image P2, analysis based on the following formulas (1) to (6) is performed. In the formula, Min () is the minimum value among the values in the parentheses, and Max () is the maximum value among the values in the parentheses.

  Next, based on the results of the above formulas (1) to (6), it is determined whether the test pattern (first test pattern TP1) can be embedded in the first image P1. That is, when the first image P1 and the second image P2 are images that satisfy all of the following expressions (7) to (9), the image light of the first projection image PA including the first image P1 Of these, the first test pattern TP1 can be formed using green light. When the first image P1 and the second image P2 do not satisfy the expressions (7) to (9), the image data of the first image P1 and the second image P2 is used as projection image data without embedding the test pattern. .

  From the above formulas (7) to (9), whether there is a brightness difference between the region in which the first test pattern TP1 is embedded in the first image P1 and the first test pattern TP1, or the embedded test pattern is recognized. Determine if you can.

  At this time, if the image signal of the first test pattern TP1 to be embedded is Test1 (x, y), the foreground portion (pattern Xa) is represented by the equation (10) and the background as a G light signal for displaying the first test pattern TP1. For the portion (pattern Xb), a signal represented by equation (11) can be adopted.

  Similarly, the R light and B light signals can be expressed by the following equations (12) and (13).

  Next, based on the image signal of the first test pattern TP1 represented by the above formulas (10) to (13) and the image data of the first image P1, the image of the first projection image PA including the first image P1. Data (first projection image data) can be obtained. Assuming that the image signal of the first projection image PA projected from the first projector is Image_PJ1 (x, y), the image signal in the area where the first test pattern TP1 is not displayed is the following equation (14), the first test: The image signal of the area where the pattern TP1 is displayed can be expressed by the following equation (15).

  Further, if the image signal of the second test pattern TP2 is Test2 (x, y), the second test pattern TP2 has a lightness complementary to that of the first test pattern TP1, and therefore the following equation (16). Can be represented.

  Therefore, if the image signal of the second projection image PB projected from the second projector is Image_PJ2 (x, y), the image signal in the area where the second test pattern TP2 is not displayed is represented by the following equation (17), The image signal of the area displaying the 2 test pattern TP2 can be expressed by the following equation (18).

  In the above description, the first test pattern TP1 is embedded in the first image P1, the first projection image is calculated, and the second projection image is obtained based on the calculation. However, the first test pattern TP1 is similarly calculated. Is embedded in the second image P2, and a paired image can be obtained based on the calculated image.

[Correction of projected image position]
Next, using the first projection image data and the second projection image data, the first projection image and the second projection image are projected onto the projection surface S from the first projector 21 and the second projector 22 shown in FIG. Projection is performed, the relative position of each projection image is obtained from the position of the first test pattern included in each projection image, and position correction is performed if necessary.

  For example, first, a first projection image in which the first test pattern TP <b> 1 is embedded from the first projector 21 is projected from the second projector 22 to form a second projection image that forms a pair. As described above, the first projector 21 forms image light using s-polarized light, and the second projector 22 forms image light using p-polarized light.

  Then, the projection image P formed on the projection surface S is captured using the digital camera 4. At this time, if the polarizing element provided in the optical path in the digital camera 4 is arranged so as to transmit the s-polarized light, the digital camera 4 captures only the first projection image in which the first test pattern TP1 is embedded. can do.

  The imaged data is sent to the image processing device 3, and the position (coordinates) of the pattern Xa included in the first test pattern TP1 is calculated. The obtained position of the first test pattern TP1 is the projection display position of the first projector 21, and the coordinates are represented by (x, y) = (Pos_PJ1x, Pos_PJ1y).

  Next, a second projection image in which the first test pattern TP1 is embedded from the second projector 22 is projected as a pair of first projection images from the first projector 21, and an internal polarization element is transmitted so as to transmit p-polarized light. An image is picked up using the digital camera 4 that controls the above. Similarly, the imaged data is sent to the image processing apparatus 3, and the position of the pattern Xa included in the first test pattern TP1 is calculated. The obtained position of the first test pattern TP1 is set as the projection display position of the second projector 22, and the coordinates are represented by (x, y) = (Pos_PJ2x, Pos_PJ2y).

  The image processing apparatus 3 calculates a position correction amount from the obtained projection positions of the first and second projectors. Assuming that the correction amount in the horizontal direction (x direction) in the in-plane direction of the projection surface S is Mx and the correction amount in the vertical direction (y direction) is My, each correction amount is expressed by the following equations (19) and (20). be able to.

  In the present embodiment, since the first projection image PA and the second projection image PB are projected and displayed with a shift of 0.5 pixels, the 0.5 pixel is added according to the term to which 0.5 in the formula is added. The amount of displacement is reflected in the position correction.

  The obtained correction amount is transmitted as a drive signal to the projection position control mechanism of the projection lens 29 of the second projector 22, and the projection position of the second projection image PB is corrected. In the image display system 1A, by appropriately performing such position correction, it is possible to display the projected image P that is well aligned.

  According to the image display system 1A having the above-described configuration, the test pattern for correcting the position of the first projection image PA and the second projection image PB is displayed without causing the user X to recognize the position, thereby correcting the displacement. It is possible to provide an image display system capable of performing the above. Therefore, when the relative position between the first projection image PA and the second projection image PB changes with time due to driving heat generation inside the projector due to use, thermal deformation of the optical member due to temperature change in the use environment, or the like. Even when the image display system 1A is being used, the relative position of the image projection position can be readjusted.

  Further, according to the image processing apparatus 3 having the above-described configuration, it is possible to satisfactorily create projection image data in which the brightness of the test pattern is complementarily changed. Furthermore, in the image processing method as described above, it is possible to create two projection image data in which the brightness of the test pattern is complementarily different.

  In the present embodiment, the projection lens 29 of the second projector 22 has the projection position control mechanism, but of course the projection lens of the first projector 21 may have the projection lens of both projectors. May have a projection position control mechanism. In addition, instead of the projection position control mechanism of the projection lens, for example, a mechanism for tilting the angle of the top plate of the mounting table is provided on the table on which the projector is mounted, and the projection position is changed by changing the installation posture of the projector itself. It doesn't matter if you do.

  In the present embodiment, the first test pattern TP1 is formed using the G light of the image light. However, the present invention is not limited to this, and the first test pattern TP1 is formed using the R light or the B light. It doesn't matter.

  However, since the human eye has the highest visibility in green, when the position of the first projection image PA and the second projection image PB is corrected with reference to green, the positional deviation of each projection image is recognized by the human eye. It becomes difficult and preferable.

  In the present embodiment, the first test pattern TP1 having one pattern Xa is used. However, the first test pattern TP1 may include a plurality of patterns Xa.

  FIG. 7 is a diagram for explaining a modification of the test pattern. Here, as shown in FIG. 7A, a first test pattern TP1 including two patterns Xa is taken as an example. In this case, the second test pattern TP2 to be paired is as shown in FIG.

  When such a test pattern is used, the relative position between the first projection image PA and the second projection image PB is detected based on the coordinates of the two feature points obtained from the two patterns Xa. Then, for example, as shown in FIG. 7C, the first projection image PA and the second projection image PB are not only shifted in the horizontal and vertical directions on the projection surface but also tilted in the projection surface. In this case, the coordinates of two feature points are obtained from the first test pattern TP1 attached to the first projection image PA and the second test pattern TP2 attached to the second projection image PB, respectively, and two points in each projection image are obtained. By detecting the inclination of the line segments connecting the two, the inclination of the projected images can be detected. Therefore, more accurate position correction can be performed.

  In addition, the same effect can be obtained by using the first test pattern TP1 including the pattern Xa having a shape (for example, a rectangle) from which two or more feature points can be obtained.

  In the present embodiment, only one area AR where the test pattern is provided is set. However, the present invention is not limited to this, and the test pattern may be provided at a plurality of places. For example, as shown in FIG. 8, when test patterns are provided in four areas AR set at the four corner positions of the projected image P, the horizontal and vertical directions of the first projected image PA and the second projected image PB are set. In addition to being displaced, it is possible to detect the inclination, and more accurate position correction can be performed.

[Second Embodiment]
FIG. 9 is an explanatory diagram of an image display system 1B according to the second embodiment of the present invention. The image display system 1B of the present embodiment is partially in common with the image display system 1A of the first embodiment. The difference is that the color of light forming the first projection image is different between the first projection image PA and the second projection image PB. Therefore, in this embodiment, the same code | symbol is attached | subjected about the component which is common in 1st Embodiment, and detailed description is abbreviate | omitted.

  As shown in FIG. 9, the image display system 1B of this embodiment has two digital cameras 41 and 42 as imaging means. The digital camera 41 has a polarizing element that transmits s-polarized light in the optical path to the photoelectric conversion element, and the digital camera 42 has a polarizing element that transmits p-polarized light in the optical path to the photoelectric conversion element. Yes. Therefore, the digital camera 41 can selectively capture only the first projection image PA, and the digital camera 42 can selectively capture only the second projection image PB.

[Image display system functions]
In the image display system 1B according to this embodiment, the first test pattern TP1 and the second test pattern TP2 are embedded in the original data of the projection image as follows, and the projection of the first projection image PA and the second projection image PB obtained is obtained. Correct the position.

[Image processing in image processing apparatus]
First, image data indicating the first image P1 and the second image P2 that do not have a test pattern is input to the image processing apparatus 3, and the test pattern of the first image P1 and the second image P2 is displayed. It is determined whether a test pattern can be added. Here, a case is shown in which it is determined that the test pattern is embedded in the R light of the image light constituting the first projection image PA and the B light of the image light constituting the second projection image PB.

  Specifically, in the area corresponding to the area AR in the projection image P of the first image P1, the analysis based on the above-described equations (1) and (2) is performed, and the area AR in the projection image P of the second image P2 Are analyzed based on the above formulas (5) and (6).

  Next, based on the analysis result, it is determined whether the test pattern can be embedded in the first image P1 and the second image P2. That is, when the first image P1 and the second image P2 are images that satisfy all of the following expressions (21) and (22), the image light of the first projection image PA including the first image P1 Among them, the first test pattern TP1 can be formed using the R light, and the first test pattern TP1 is formed using the B light out of the image light of the second projection image PB including the second image P2. Can do. In order not to satisfy the expressions (21) and (22), the image data of the first image P1 and the second image P2 are set as projection image data without embedding the test pattern.

  At this time, if the image signal of the first test pattern TP1 embedded in the R light of the first image P1 is TestR (x, y), the foreground portion (pattern Xa) is used as the R light signal for displaying the first test pattern. Can adopt the signal expressed by Equation (23) and the background portion (pattern Xb) expressed by Equation (24).

  Similarly, if the image signal of the first test pattern TP1 embedded in the B light of the second image P2 is TestB (x, y), the foreground portion (pattern Xa) is used as the R light signal for displaying the first test pattern. The signal expressed by the equation (25) and the background portion (pattern Xb) by the equation (26) can be used.

  Then, based on the image signal of each first test pattern represented by the above formulas (23) to (26) and the image data of the first image P1 or the image data of the second image P2, the first projection image data And the 2nd projection image data can be calculated | required. In the first projection image data, an image signal in a region where the first test pattern TP1 is not displayed is represented by the following equations (27) to (29), and an image signal in the region where the first test pattern TP1 is displayed is represented by the following equation (30). To (32).

  Similarly, in the second projection image data, the image signal in the area where the first test pattern TP1 is not displayed is represented by the following formulas (33) to (35), and the image signal in the area where the first test pattern TP1 is displayed is represented by the following formula. (36) to (38).

[Correction of projected image position]
Next, the first projection image PA and the second projection image PB represented by the above formulas (27) to (38) are projected from the first projector 21 and the second projector 22 shown in FIG. The first projection image and the second projection image are projected onto the image, the feature point is obtained from the position of the first test pattern included in each projection image, the relative position of each projection image is obtained, and position correction is performed if necessary. .

  That is, the first projected image PA and the second projected image PB are projected onto the projection surface S to form the projected image P, and the projected image P is captured using the digital camera 4 (41, 42). At this time, the digital camera 41 acquires only the first projection image PA, which is an s-polarized image, for each RGB component via the polarizing element. Similarly, the digital camera 42 acquires only the p-polarized image for each RGB component via the polarizing element.

  The imaged data is sent to the image processing device 3, and the position (coordinates) of the pattern Xa of the first test pattern included in each projection image is calculated. The calculated coordinates are used as the projection display positions of the projectors that project each projection image, and the position correction amount is calculated from the obtained projection positions of the first and second projectors.

  The obtained correction amount is transmitted as a drive signal to the projection position control mechanism of the projection lens 29 of the second projector 22, and the projection position of the second projection image PB is corrected. In the image display system 1A, by appropriately performing such position correction, it is possible to display the projected image P that is well aligned.

  According to the image display system 1B configured as described above, the two digital cameras 41 and 42 can simultaneously capture the first test pattern included in the first projection image PA and the second projection image PB. Thus, an error caused by a difference in imaging timing, such as a measurement error due to vibration of the projector, does not occur, and the accuracy of position correction can be increased.

  In addition, when the pixels of the first projection image PA and the second projection image PB are superimposed with a shift of 0.5 pixels from each other, the pattern for positioning is displayed with red and blue color lights, thereby forming the green color light. The resulting image maintains a high resolution, and position correction is possible for other colors. Therefore, it is possible to display a higher quality image.

  In the present embodiment, the test pattern is embedded in the R light of the first projection image PA and the B light of the second projection image PB, but the first projection image PA and the second projection image PB If the test pattern is embedded in the light of different colors, other combinations can be selected.

[Third Embodiment]
10 and 11 are explanatory diagrams of the projector 23 included in the image display system according to the third embodiment of the present invention. The image display system of the present embodiment is partially in common with the image display systems of the first and second embodiments. The difference is that the image display system of the first and second embodiments has two projectors, whereas the projector 23 of the image display system of the present embodiment has two image forming optical systems (image forming engines). The two image forming optical systems cooperate to display one image. Therefore, in this embodiment, the same code | symbol is attached | subjected about the component which is common in 1st Embodiment, and detailed description is abbreviate | omitted.

  FIG. 10 is a schematic configuration diagram of the projector according to the present embodiment. The projector 23 according to the present embodiment is a so-called six-plate projector that includes six sets of liquid crystal light valves, two sets of three colors of R, G, and B. In each of the following drawings, in order to make the components easy to see, the ratios and scales of the dimensions may be varied depending on the components.

  As shown in the figure, the projector 23 modulates light emitted from a light source according to an input image signal, and forms two image forming optical systems 12A and 12B that form image light, and the image forming optical systems 12A and 12A, And a projection optical system 13 that projects the image light formed by 12B.

  The image forming optical systems 12A and 12B include an image forming optical system 12A arranged so that the emission direction of the formed image light and the projection direction of the image light by the projection optical system 13 are substantially parallel to each other, and the formed image light. The image forming optical system 12B is disposed so that the emission direction and the projection direction of the image light by the projection optical system 13 are substantially orthogonal to each other. Each of the image forming optical systems 12A and 12B includes an illumination optical system 14, a color separation optical system 15, a relay optical system 16, and electro-optical devices 17A and 17B.

  The illumination optical system 14 forms an image of light emitted from the light source 19 in the illuminated area, and includes the light source 19, the first lens array 111, the second lens array 112, and the superimposing lens 113. ing. Although not shown, the light source 19 is a light source lamp that emits a radial light beam, a reflector that reflects the emitted light emitted from the light source lamp and converges it at a predetermined position, and the light converged by the reflector is an illumination optical axis. And a collimating concave lens that collimates with respect to A.

  The first lens array 111 and the second lens array 112 have a configuration in which small lenses corresponding to each other are arranged in a matrix, and the first lens array 111 transmits light incident from the light source 19 into a plurality of portions. The light is divided and imaged in the vicinity of the second lens array 112. In addition, the second lens array 112 forms an image emitted from each small lens of the first lens array 111 together with the superimposing lens 113 positioned in the subsequent stage of the optical path, on the liquid crystal panel of the electro-optical devices 17A and 17B described later. Image the area.

  Each partial light emitted from the second lens array 112 is linearly polarized light having substantially one type of polarization direction by a polarization conversion element (not shown) disposed between the second lens array 112 and the superimposing lens 113. Is converted to Further, the linearly polarized light converted by the polarization conversion element in the image forming optical system 12A and the linearly polarized light converted by the polarization conversion element in the image forming optical system 12B are linearly polarized light having different types of polarization directions. Yes.

  The color separation optical system 15 includes two dichroic mirrors 115 and 116 and a reflection mirror 117, and the dichroic mirrors 115 and 116 convert a plurality of partial lights emitted from the illumination optical system 14 into red light ( R light), green light (G light), and blue light (B light). The relay optical system 16 includes an incident side lens 119, a relay lens 120, and reflection mirrors 121 and 122. The light is separated by the color separation optical system 15, and blue light having a longer optical path than other color light is used for blue light. The liquid crystal light valve 124B has a function to guide the liquid crystal light valve 124B.

  At this time, the dichroic mirror 115 of the color separation optical system 15 transmits the red light component of the light emitted from the illumination optical system 14, while reflecting the green light component and the blue light component. The red light transmitted through the dichroic mirror 115 is reflected by the reflection mirror 117, passes through the field lens 125, and reaches the liquid crystal light valve 124R for red light. The field lens 125 converts each partial light emitted from the second lens array 112 into light parallel to the central axis (principal ray). The same applies to the field lens 125 provided on the light incident side of the liquid crystal light valve 124G for green light and the liquid crystal light valve 124B.

  Of the green light and blue light reflected by the dichroic mirror 115, the green light is reflected by the dichroic mirror 116, passes through the field lens 125, and reaches the liquid crystal light valve 124G. On the other hand, the blue light passes through the dichroic mirror 116, passes through the relay optical system 16, and further passes through the field lens 125 to reach the liquid crystal light valve 124B.

  The electro-optical devices 17A and 17B modulate incident light in accordance with an image signal to form image light. The emission direction of the formed image light and the projection direction of the image light by the projection optical system 13 are substantially the same. The electro-optical device 17A arranged so as to be parallel to each other, and the electro-optical device 17B arranged so that the emission direction of the formed image light and the projection direction of the image light by the projection optical system 13 are substantially orthogonal to each other. Has been.

  Each of the electro-optical devices 17A and 17B includes three liquid crystal light valves, an incident side polarizing plate (not shown) disposed on the light incident side of each of the liquid crystal light valves 124R, 124G, and 124B, and the light of each liquid crystal light valve. An exit-side polarizing plate (not shown) disposed on the exit side and a cross dichroic prism 126 (color synthesis optical system) are provided.

  The incident-side polarizing plate transmits only polarized light having a polarization direction substantially the same as the polarization direction aligned by the polarization conversion element among the color lights separated by the color separation optical system 15 and absorbs other light. The liquid crystal light valves 124R, 124G, and 124B as light modulation elements control the polarization direction of polarized light incident from the incident-side polarizing plate by controlling the alignment state of the liquid crystal in the liquid crystal panel according to the input image signal. Modulate. The exit-side polarizing plate transmits polarized light in a certain direction (for example, light having a polarization axis perpendicular to the light transmission axis of the incident-side polarizing plate) out of the light emitted through the liquid crystal panel, and other light. Absorbs.

  A cross dichroic prism 126 as a color synthesis optical system combines the color lights emitted from the emission-side polarizing plates to form image light. The cross dichroic prism 126 has a substantially square shape in plan view in which four right angle prisms are bonded together, and two dielectric multilayer films are formed on the interface where the right angle prisms are bonded together. These dielectric multilayer films transmit green light emitted from the liquid crystal light valve 124G and reflect red light and blue light emitted from the liquid crystal light valves 124R and 124B. Thereby, image light (color image) in which red light, green light, and blue light are combined is formed. In the image forming optical system 12A, image light indicating the first projection image PA is formed, and in the image forming optical system 12B, image light indicating the second projection image PB is formed.

  The projection optical system 13 includes a polarized beam splitter (hereinafter abbreviated as PBS) 131 and a projection lens 132. The PBS 131 is configured by bonding two right-angle prisms, and transmits light in one polarization direction and reflects light in the other polarization direction through a dielectric polarizing film formed at the interface where the right-angle prisms are bonded. Let

  In the present embodiment, the image light indicating the first projection image PA emitted from the image forming optical system 12A passes through the dielectric polarizing film, and the image light indicating the second projection image PB emitted from the image forming optical system 12B. Are reflected by the dielectric polarizing film and guided to the projection lens 132, respectively. The projection lens 132 is configured as a combined lens in which a plurality of lenses are housed in a cylindrical lens barrel (not shown), and enlarges and projects image light emitted from the PBS 131 onto a projection surface S such as a screen.

  Further, as shown in FIG. 11, the projector 23 includes a position correction unit 5 that shifts the position of the entire image forming optical system 12 </ b> B. Based on the position correction amount calculated by the image processing apparatus 3, the image formation is performed. The projection position of the second projection image PB on the projection surface S can be changed by moving the optical system 12B.

  According to the image display system having the projector 23 having the above-described configuration, since only one projector is used, the configuration of the apparatus can be simplified, and the first projection image and the first projection image can be reliably corrected by correcting the positional deviation. An image display system for projecting and displaying the second projected image can be provided.

  The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but it goes without saying that the present invention is not limited to such examples. Various shapes, combinations, and the like of the constituent members shown in the above-described examples are examples, and various modifications can be made based on design requirements and the like without departing from the gist of the present invention.

  In the above embodiment, the image display system displays the high-definition projection image P by superimposing the first projection image PA and the second projection image PB with a shift of 0.5 pixels. However, the present invention is not limited to this, and the high-brightness projection image P may be obtained by projecting and displaying the pixels of the first projection image PA and the second projection image PB in an overlapping manner. Alternatively, the peripheral portions of the first projection image PA and the second projection image PB may be partially overlapped with each other so that the two projection images cooperate to form one large projection image.

  Regardless of which display method the image display system employs, by using the present invention, a test pattern for correcting the position of the first projection image PA and the second projection image PB can be obtained without causing the user X to recognize the pattern. It is possible to display and correct the positional deviation.

  In addition, when the projector to be used has a deviation in the RGB image display position due to assembly variations at the time of manufacture, aberration of the projection lens, or the like, a deviation in the position of the image drawn by the R light with respect to the image drawn by the G light in advance. In addition, the positional deviation of the image drawn by the B light with respect to the image drawn by the G light is measured, and the true position correction amount is calculated from the position correction amount detected by the present invention and the position deviation amount measured in advance. It may be calculated as

  DESCRIPTION OF SYMBOLS 1A, 1B ... Image display system, 3 ... Image processing apparatus, 4 ... Digital camera (imaging means), 5 ... Position correction means, 12A ... 1st image forming optical system, 12B ... 2nd image forming optical system, 13 ... Projection Optical system, 23 ... projector, 29 ... projection lens (projection optical system, position correction means), 41 ... digital camera (first imaging means), 42 ... digital camera (second imaging means), P ... projection image (predetermined) Image), PA ... first projection image, PB ... second projection image, P1 ... first image, P2 ... second image, S ... projection surface, TP1 ... first test pattern (first pattern), TP1 ... first 2 test patterns (second pattern),

Claims (8)

The first projection image data indicating the first projection image and the second projection image input to an image display system that displays a predetermined image by projecting the first projection image and the second projection image in an overlapping manner are shown. An image processing apparatus for creating second projection image data,
The first projection image is an image obtained by adding a first pattern for positioning to the first image,
The second projected image is an image obtained by adding a second pattern corresponding to the first pattern to the second image;
Based on the first image data indicating the first image and the second image data indicating the second image, the brightness obtained by adding the brightness of the first pattern and the brightness of the second pattern is the predetermined value. The first pattern image data and the second pattern image data are complementarily made different from each other so that the brightness of the corresponding portion of the image is the same as the brightness of the corresponding portion, and the first projection image data and the second projection image data are An image processing apparatus characterized by being created. As the first projected image, an image obtained by adding the first pattern to the first image and an image obtained by adding the first pattern to the second image are obtained, and the second projected images that are paired with each other Seeking
The first pattern included in an image obtained by adding the first pattern to the first image and an image obtained by adding the first pattern to the second image is defined using green light. The image processing apparatus according to claim 1. The first projected image and the second projected image are projected on the projection surface while being shifted from each other by half a pixel,
As the first projected image, an image obtained by adding the first pattern to the first image and an image obtained by adding the first pattern to the second image are obtained, and the second projected images that are paired with each other Seeking
The first pattern attached to the first image is defined using one of red light and blue light;
The image processing apparatus according to claim 1, wherein the first pattern attached to the second image is defined using the other of red light and blue light.   The image processing apparatus according to claim 1, wherein the first pattern has a shape in which two or more coordinates are specified. An image processing apparatus according to any one of claims 1 to 4,
A first image forming optical system that forms one of the first projected image and the second projected image using s-polarized light based on projection image data supplied from the image processing device;
A second image forming optical system for forming the other image of the first projection image or the second projection image using p-polarized light based on the projection image data supplied from the image processing device;
A projection optical system that displays the predetermined image by superimposing the first projection image and the second projection image on a projection surface;
A polarizing element that transmits either s-polarized light or p-polarized light, and when observing the predetermined image, the first projected image and the second projected image projected on the projection surface An imaging means for selectively imaging one of the projected images,
Based on the projection positions of the first projection image and the second projection image on the projection surface, which are obtained from the captured images of the first projection image and the second projection image captured by the imaging unit, Position correction for moving at least one of the first projection image and the second projection image so that the relative position between the first projection image and the second projection image approaches a predetermined target position. And an image display system.   The said imaging means has a 1st imaging means which has a polarizing element which permeate | transmits s-polarized light, and a 2nd imaging means which has a polarizing element which permeate | transmits p-polarized light, The said imaging means is characterized by the above-mentioned. Image display system.   The first image forming optical system, the second image forming optical system, and a projection optical system that projects the first projection image and the second projection image so as to overlap each other on the projection surface. The image display system according to claim 6, further comprising a projector. The first projection image data indicating the first projection image and the second projection image input to an image display system that displays a predetermined image by projecting the first projection image and the second projection image in an overlapping manner are shown. An image processing method for creating second projection image data,
The first projection image is an image obtained by adding a first pattern for positioning to the first image,
The second projected image is an image obtained by adding a second pattern corresponding to the first pattern to the second image;
Based on the first image data indicating the first image and the second image data indicating the second image, the brightness obtained by adding the brightness of the first pattern and the brightness of the second pattern is the predetermined value. The first pattern image data and the second pattern image data are complementarily made different from each other so that the brightness of the corresponding portion of the image is the same as the brightness of the corresponding portion, and the first projection image data and the second projection image data are An image processing method characterized by creating.

Images