JP6624942B2 - Projection system, projector device, and program - Google Patents

Description

  The present invention relates to a technique for appropriately displaying an image (video) projected by a projection type projector device.

  Various techniques have been developed for correcting geometric distortion that occurs when an image (video) projected by a projection-type projector device is viewed from a user's viewpoint.

  For example, Patent Literature 1 discloses an image projection device capable of correcting image distortion caused by distortion of a screen itself. In this video projection apparatus, a test image is projected from a projector onto a screen, the projected test image is captured by a camera-equipped mobile phone, and geometric correction is performed based on the image captured by the camera-equipped mobile phone. Thus, in the image projection device, image distortion (geometric distortion) due to distortion of the screen itself can be corrected.

JP 2006-33357 A

  However, in the above-described related art, since a geometric correction is performed based on an image captured by a camera-equipped mobile phone, a device having a shooting function such as a camera-equipped mobile phone is separately required. When geometric correction of an image projected on a screen by a projection type projector device is performed without using a device having such a photographing function, for example, the user is required to move four vertices of a rectangular pattern projected on the screen. It is conceivable to adjust the projection state of the projection type projector device so that a rectangular pattern without geometric distortion is projected. That is, it is conceivable that the user performs four-point geometric correction. Such a four-point geometric correction is difficult for a general user to operate, and it is difficult to adjust the projection-type projector apparatus so that projection in a state without completely geometric distortion is realized.

  In general, the projection target of the projection type projector apparatus is often a flat screen that is installed vertically. For this reason, even if projection in a state without completely geometric distortion is not realized, the user adjusts the projection-type projector apparatus to a projection state in which the user does not mind the geometric distortion of the projected image. It is possible enough. This is because the user can correctly recognize that the image is projected on the flat screen that is installed vertically, and it is easy to correctly recognize the vertical direction. In addition, even if there is some geometric distortion, the user does not feel uncomfortable.

  However, when the projection target (projection surface) is, for example, an inclined plane (for example, an inclined ceiling) inclined at a predetermined angle installed above the user, and an image is projected from the projection-type projector device onto the inclined plane, When the projection plane is a vertical plane, it is necessary to further reduce the allowable level of geometric distortion. This is because, when the projection plane is an inclined plane (for example, an inclined ceiling), it is not easy for the user to intuitively recognize the vertical direction of the projected image, and therefore, a sense of discomfort is felt in the image projected on the vertical plane. This is because, in the image projected on the inclined plane (for example, the inclined ceiling), the level of the geometric distortion at which the image is not generated makes the user feel uncomfortable.

  Therefore, the present invention provides a projection system, a projector device, and a camera that easily and appropriately reduce geometric distortion of an image projected on an inclined plane (for example, an inclined ceiling) without using a device having a photographing function. , To realize the program.

In order to solve the above problem, a first invention is directed to a plane parallel to a plane including a left eye viewpoint and a right eye viewpoint, and a common tangent plane of the right eye cornea and the left eye cornea. On the other hand, a projection system that projects an image such that an inclined plane is used as a projection surface and geometric image distortion is reduced when viewed from a user's viewpoint, comprising: a projection unit; A shape measuring unit, a controller, and a projected image adjusting unit are provided.
The projection unit is a projection unit that projects an image on a projection surface, and projects a first test image having a vertical reference line that defines a vertical direction of the image, and has a horizontal reference line that defines a horizontal direction of the image. Project a second test image.

  The three-dimensional shape measuring unit measures a three-dimensional shape of the projection plane.

  The controller generates a control signal based on a user operation.

The projection image adjustment unit performs the following processing.
(1A) Based on the three-dimensional shape data measured by the three-dimensional shape measurement unit, the projection image adjustment unit passes through the projection center point, which is the intersection of the projection axis of the projection unit and the projection surface, and is on the normal to the projection surface. When the first test image is viewed from the first point, a correction process, which is an image conversion process, is performed on the first test image so that the geometric image distortion is reduced.
(2A) The projection image adjustment unit determines the first rotation angle according to the control signal, and, when viewing the projection plane from the first point, projects the corrected first test image onto the projection plane by the projection unit. Performs a first rotation process, which is an image conversion process, on the first test image after the correction process so that the image is rotated by the first rotation angle around the projection center point.
(1B) Based on the three-dimensional shape data measured by the three-dimensional shape measurement unit, the projection image adjustment unit passes through the projection center point, which is the intersection of the projection axis of the projection unit and the projection surface, and is on the normal to the projection surface. When viewed from the first point, which is the point (1), a correction process, which is an image conversion process, is performed on the second test image so that the geometric image distortion is reduced.
(2B) The projection image adjustment unit determines the second rotation angle according to the control signal, and when the projection surface is viewed from the first point, the projection image of the second test image after the correction processing is projected onto the projection surface by the projection unit. Performs a second rotation process, which is an image conversion process, on the second test image after the correction process so that the image is rotated by the second rotation angle about the projection center point.
(3) The projection image adjustment unit includes a first straight line including a vertical reference line on the projection plane of the first test image after the first rotation processing, and a projection plane of the second test image after the second rotation processing. Based on a second straight line including the upper vertical reference line, a viewpoint candidate point where the first straight line and the second straight line appear orthogonal to each other is determined, and when viewed from the determined viewpoint candidate point, a geometrical point is determined. A viewpoint specifying process for converting an image projected on the projection plane is performed so that a typical image distortion is reduced.

  In this projection system, the viewpoint specifying process is performed as described above, and an image without geometric distortion (an image with reduced geometric distortion) when viewed from the user's viewpoint is converted into, for example, an inclined ceiling plane. Can be projected.

  Therefore, in this projection system, it is possible to easily and appropriately eliminate (reduce) geometric distortion of an image projected on an inclined projection surface (for example, an inclined ceiling) without using an apparatus having a photographing function. it can.

  A second invention is the first invention, wherein the projection image adjustment unit performs the following processing.

In the xyz coordinate space in which the x-axis and the y-axis are set such that the optical axis of the projection unit is the z-axis, and the plane having the optical axis of the projection unit as the normal is the xy plane,
A point on the first straight line which is a straight line including the vertical reference line on the projection plane of the first test image after the first rotation processing is defined as a point Pa, and coordinates of the point Pa are defined as (ax, ay, az).
A point on the second straight line that is a straight line including the horizontal reference line on the projection plane of the second test image after the second rotation processing is a point Pb, and coordinates of the point Pb are (bx, by, bz);
When the intersection of the first straight line and the second straight line is calculated as a point Pc, and the coordinates of the calculated point Pc are (cx, cy, cz),
A = −ax × by + ay × bx
B = −ax × bz + az × bx
C = -az × by + ay × bz
Is performed, and coefficients A, B, and C are obtained.

により、回転行列Ｒｘ，Ｒｙ、Ｒｚを取得する。 The projection image adjustment unit

To obtain rotation matrices Rx, Ry, and Rz.

The projection image adjustment unit calculates a composite matrix of the rotation matrices Rx, Ry, and Rz as R
R = Rz ・ Ry ・ Rx
And the inverse matrix of the rotation matrix is obtained as R ⁻¹ .

The projection image adjustment unit obtains, as the point Pb ′, a point converted by the synthesis matrix with respect to the coordinates (bx, by, bz) of the point Pb, and sets the coordinates of the point Pb ′ to (bx ′, by ′, bz ′). ) And get
The angle θ is determined based on a control signal from the controller.

により取得し、
視点候補点の座標（ｘ，ｙ，ｚ）を、

Ｖ’＝（０，０，１）
により算出する。 The projection image adjustment unit calculates the coefficient A ′

Acquired by
The coordinates (x, y, z) of the viewpoint candidate point are

V '= (0,0,1)
It is calculated by:

  In this projection system, the viewpoint specifying process is performed as described above, and an image without geometric distortion (an image with reduced geometric distortion) when viewed from the user's viewpoint is converted into, for example, an inclined ceiling plane. Can be projected.

  Therefore, in this projection system, it is possible to easily and appropriately eliminate (reduce) geometric distortion of an image projected on an inclined projection surface (for example, an inclined ceiling) without using an apparatus having a photographing function. it can.

  A third invention is the first or second invention, wherein the first test image includes a vertical reference line passing through a center point of the first test image while being displayed on the display screen.

  This makes it easy for the projection system to specify the first straight line including the vertical reference line of the first test image on the projection plane.

  A fourth invention is the first or the second invention, wherein the second test image includes a horizontal reference line passing through a center point of the second test image when displayed on the display screen.

  This makes it easy for the projection system to specify the first straight line including the horizontal reference line of the second test image on the projection plane.

  A fifth invention is the invention according to any one of the first to fourth inventions, wherein the first test image is an image including a pattern capable of discriminating an upper part of the image while being displayed on the display screen. It is.

  Thereby, in this projection system, the upper and lower sides of the projection image on the projection plane can be easily specified.

  A sixth invention is the invention according to any one of the first to fifth inventions, wherein the projection unit, when the viewpoint specifying process is performed, includes a third test image including a grid-like pattern including a plurality of square grid patterns. Is projected.

  Thus, in this projection system, the degree of distortion of the lattice pattern can be easily grasped, and the degree of distortion of the projected image can be easily recognized.

  In a seventh aspect based on the sixth aspect, the third test image is configured such that the first lattice pattern having the first pattern and the second lattice pattern having the second pattern have a geometric distortion. In the state where there is no image, the image includes a grid pattern formed by being alternately arranged in the first direction on the test image and in the second direction orthogonal to the first direction on the test image.

  Thus, in this projection system, the degree of distortion of the lattice pattern can be easily grasped, and the degree of distortion of the projected image can be easily recognized.

  An eighth invention is a projector device used in the projection system according to any one of the first to seventh inventions, and includes a projection unit and a projection image adjustment unit.

  Thereby, a projector device used in the projection system according to any one of the first to seventh inventions can be realized.

  A ninth invention is a projection method executed by using a controller that generates a control signal based on a user operation, and is a plane parallel to a plane including a left-eye viewpoint and a right-eye viewpoint, and With respect to the plane which is a common tangent plane of the cornea and the left eye cornea, an image is formed such that an inclined plane is a projection plane and a geometric image distortion is reduced when viewed from a user's viewpoint. Is a program for causing a computer to execute a projection method for projecting the image.

  The projection method includes a projection step, a three-dimensional shape measurement step, a controller, and a projection image adjustment step.

  The projecting step is a projecting step of projecting an image on a projection surface, and projects a first test image having a vertical reference line defining a vertical direction of the image, and has a horizontal reference line defining a horizontal direction of the image. Project a second test image.

  The three-dimensional shape measuring step measures a three-dimensional shape of the projection plane.

The projection image adjustment step executes the following processing.
(1A) The projection image adjustment step passes a projection center point, which is an intersection of a projection axis for projecting an image in the projection step and a projection plane, based on the three-dimensional shape data measured in the three-dimensional shape measurement step. When viewed from the first point which is a point on the normal to the projection plane, a correction process, which is an image conversion process, is performed on the first test image so that the geometric image distortion is reduced. Execute.
(2A) The projection image adjusting step determines the first rotation angle according to the control signal, and, when the projection surface is viewed from the first point, the projection image on the projection surface in the projection step of the first test image after the correction processing. Performs a first rotation process, which is an image conversion process, on the first test image after the correction process so that the image is rotated by the first rotation angle around the projection center point.
(1B) The projection image adjustment step passes a projection center point, which is an intersection of a projection axis for projecting an image in the projection step and a projection plane, based on the three-dimensional shape data measured in the three-dimensional shape measurement step. In addition, when viewed from a first point which is a point on the normal to the projection plane, a correction process, which is an image conversion process, is performed on the second test image so that the geometric image distortion is reduced. Execute.
(2B) The projection image adjustment step determines the second rotation angle according to the control signal, and, when the projection plane is viewed from the first point, the projection image on the projection plane in the projection step of the second test image after the correction processing. Performs a second rotation process, which is an image conversion process, on the second test image after the correction process so that the image is rotated by the second rotation angle about the projection center point.
(3) The projection image adjustment step includes: a first straight line including a vertical reference line on the projection plane of the first test image after the first rotation processing; and a projection plane of the second test image after the second rotation processing. Based on a second straight line including the upper vertical reference line, a viewpoint candidate point where the first straight line and the second straight line appear orthogonal to each other is determined, and when viewed from the determined viewpoint candidate point, a geometrical point is determined. A viewpoint specifying process for converting an image projected on the projection plane is performed so that a typical image distortion is reduced.

  Thus, it is possible to realize a program for causing a computer to execute a projection method having the same effect as the first invention.

  ADVANTAGE OF THE INVENTION According to this invention, the projection system which reduces easily and appropriately the geometric distortion of the image projected on the inclined plane (for example, inclined ceiling), without using the apparatus which has a photography function, a projector apparatus, and , The program can be realized.

FIG. 1 is a schematic configuration diagram of a projection system 1000 according to a first embodiment. FIG. 2 is a schematic configuration diagram of a projection image adjustment unit 1 of the projector device 100 of the projection system 1000 according to the first embodiment. FIG. 2 is a schematic configuration diagram of a viewpoint specification processing unit 15 of the projection system 1000 according to the first embodiment. FIG. 2 is a schematic configuration diagram of a viewpoint candidate acquisition unit 151 of the projection system 1000 according to the first embodiment. The figure which showed typically the three-dimensional space. The figure which shows the 1st test image Img1_T1. The figure which showed typically the three-dimensional space in the state 1. The figure which showed typically the three-dimensional space in the state 2. The figure which shows the 2nd test image Img1_T2. The figure for demonstrating a 2nd rotation process. The figure which showed typically the three-dimensional space in the state 4. The figure for demonstrating a viewpoint identification process. The figure which shows the 3rd test image Img1_T3. The figure showing the state when the third test image Img1_T3 is projected onto the inclined ceiling CL when the projection system 1000 executes the (N + 1) th viewpoint specification processing and specifies the viewpoint candidate point as Vp_cand (N + 1). . FIG. 4 is a diagram showing a flowchart of a projection method executed by the projection system 1000. FIG. 6 is a view showing a flowchart of viewpoint identification processing of a projection method executed by the projection system 1000. The figure for demonstrating the principle of a viewpoint specific process. The figure which shows the point A0 ', the point B0', and the point C0 'of a three-dimensional coordinate space. The figure which shows the point A0 '', the point B0 '', and the point C0 '' of a three-dimensional coordinate space. FIG. 2 is a diagram showing a CPU bus configuration.

[First Embodiment]
The first embodiment will be described below with reference to the drawings.

<1.1: Configuration of projection system>
FIG. 1 is a schematic configuration diagram of a projection system 1000 according to the first embodiment.

  FIG. 2 is a schematic configuration diagram of the projection image adjustment unit 1 of the projector device 100 of the projection system 1000 according to the first embodiment.

  FIG. 3 is a schematic configuration diagram of the viewpoint specification processing unit 15 of the projection system 1000 according to the first embodiment.

  FIG. 4 is a schematic configuration diagram of the viewpoint candidate acquisition unit 151 of the projection system 1000 according to the first embodiment.

  The projection system 1000 includes a projector device 100 and a controller 200, as shown in FIG.

  As shown in FIG. 1, the projector device 100 includes a projection image adjustment unit 1, a test image storage unit 2, a projection unit 3, a three-dimensional shape measurement unit 4, a three-dimensional shape data storage unit 5, a first An interface 6 is provided.

  As shown in FIG. 2, the projection image adjustment unit 1 includes a first selector SEL1, a first switch SW1, a correction unit 11, a second switch 12, a first rotation processing unit 13, and a second selector SEL2, a second rotation processing unit 14, a third selector SEL3, a viewpoint specification processing unit 15, a fourth selector SEL4, and a fifth selector SEL5.

  The first selector SEL1 receives an image Din (image signal Din) input to the projector device 100, a test image Img1 output from the test image storage unit 2, and a selection signal sel1. The first selector SEL1 selects one of the image Din and the test image Img1 according to the selection signal sel1, and outputs the selected image (image signal) to the first switch SW1 as the image D1 (image signal D1). Note that the selection signal sel1 is a control signal generated by a control unit (not shown) that controls each functional unit of the projector device 100.

  The first switch SW1 receives an image D1 output from the first selector SEL1 and a switching signal sw1. The first switch SW1 outputs the image D1 (image signal D1) to one of the correction unit 11 and the viewpoint identification processing unit 15 according to the switching signal sw1. Note that the switching signal sw1 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The correction unit 11 receives the image D1 output from the first switch SW1, the three-dimensional shape data 3D_data output from the three-dimensional shape data storage unit 5, and information P_prj about the projection point of the projection unit 3. I do. The correction unit 11 performs a correction process on the image D1 based on the three-dimensional shape data 3D_data and the information P_prj on the projection point. Then, the correction unit 11 outputs the image after the correction processing to the second selector SEL2 and the first rotation processing unit 13 as an image D2 (image signal D2).

  The second switch 12 receives the signal Sig1 output from the first interface 6 and the switching signal sw2. The second switch 12 outputs the signal Sig1 to any one of the first rotation processing unit 13, the second rotation processing unit 14, and the viewpoint identification processing unit 15 according to the switching signal sw2. Note that the switching signal sw2 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The first rotation processing unit 13 includes an image D2 output from the correction unit 11, three-dimensional shape data 3D_data output from the three-dimensional shape data storage unit 5, information P_prj about a projection point of the projection unit 3, The signal Sig1 output from the second switch 12 is input. When the signal Sig1 is input, the first rotation processing unit 13 executes the first rotation processing on the image D2 based on the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3. (Details will be described later). Then, the first rotation processing unit 13 outputs the processed image to the second selector SEL2 as an image D21 (image signal D21). Further, the first rotation processing unit 13 outputs a detection signal including information (detection information) when the first rotation processing is completed to the viewpoint specifying processing unit 15 as Det1 (details will be described later). The detection signal Det1 includes data of a first straight line (first straight line data) (details will be described later).

  The second selector SEL2 receives an image D2 output from the correction unit 11, an image D21 output from the first rotation processing unit 13, and a selection signal sel2. The second selector SEL2 selects one of the image D2 and the image D21 according to the selection signal sel2, and uses the selected image as an image D3 (image signal D3), the second rotation processing unit 14, and the third selector SEL3. And output to Note that the selection signal sel2 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The second rotation processing unit 14 includes an image D3 output from the second selector SEL2, three-dimensional shape data 3D_data output from the three-dimensional shape data storage unit 5, information P_prj on a projection point of the projection unit 3, The signal Sig1 output from the second switch 12 is input. When the signal Sig1 is input, the second rotation processing unit 14 performs the second rotation processing on the image D3 based on the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3. (Details will be described later). Then, the second rotation processing unit 14 outputs the processed image to the third selector SEL3 as an image D31 (image signal D31). Further, the second rotation processing unit 14 outputs a detection signal including information (detection information) when the first rotation processing is completed to the viewpoint specifying processing unit 15 as Det2 (details will be described later). The detection signal Det2 includes data of a second straight line (second straight line data) (details will be described later).

  The third selector SEL3 receives an image D3 output from the second selector SEL2, an image D31 output from the second rotation processing unit 14, and a selection signal sel3. The third selector SEL3 selects one of the image D3 and the image D31 according to the selection signal sel3, and outputs the selected image to the fourth selector SEL4 as an image D4 (image signal D4). Note that the selection signal sel3 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The viewpoint specification processing unit 15 includes an image D1 output from the first switch SW1, three-dimensional shape data 3D_data output from the three-dimensional shape data storage unit 5, information P_prj on a projection point of the projection unit 3, Enter Further, the viewpoint specification processing unit 15 receives the detection signal Det1 output from the first rotation processing unit 13 and the detection signal Det2 output from the second rotation processing unit 14.

  The viewpoint specification processing unit 15 includes a viewpoint candidate acquisition unit 151 and a projection image acquisition unit 152, as shown in FIG.

  As shown in FIG. 4, the viewpoint candidate acquisition unit 151 includes an intersection calculation unit 1511, a coefficient calculation unit 1512, a transformation matrix acquisition unit 1513, a transformation matrix synthesis unit 1514, an inverse transformation matrix acquisition unit 1515, and a transformation unit. 1515, a coefficient acquisition unit 1517, an angle acquisition unit 1518, and a viewpoint candidate point calculation unit 1519.

  The intersection calculation unit 1511 receives the detection signal Det1 output from the first rotation processing unit 13 and the detection signal Det2 output from the second rotation processing unit 14. Based on the first straight line data included in the detection signal Det1 and the second straight line data included in the detection signal Det2, the intersection calculation unit 1511 calculates the coordinates of the intersection Pc between the first straight line and the second straight line, The coordinates of a point Pa on a straight line other than the intersection Pc and the coordinates of a point Pb on the second straight line other than the intersection Pc are calculated, and information on the calculated coordinates of the intersection Pc is calculated. The included signal (data) is output to the viewpoint candidate point calculation unit 1519 as Data (Pc).

  Further, the intersection calculation unit 1511 outputs a signal (data) including information on the coordinates of the point Pa to the coefficient calculation unit 1512 and the transformation matrix acquisition unit 1513 as Data (Pa).

  Further, the intersection calculation unit 1511 outputs a signal (data) including information on the coordinates of the point Pb to the coefficient calculation unit 1512 and the conversion unit 1516 as Data (Pb).

  The coefficient calculation unit 1512 receives the data Data (Pa) of the point Pa and the data Data (Pb) of the point Pb output from the intersection calculation unit 1511. The coefficient calculation unit 1512 acquires information on the coefficients A, B, and C based on the data Data (Pa) of the point Pa and the data Data (Pb) of the point Pb (details will be described later). Then, the coefficient calculation unit 1512 outputs information (data) on the obtained coefficients A, B, and C to the transformation matrix acquisition unit 1513.

  The transformation matrix acquisition unit 1513 receives information (data) on the coefficients A, B, and C output from the coefficient calculation unit 1512 and data Data (Pa) of the point Pa output from the intersection calculation unit 1511. . The transformation matrix acquisition unit 1513 acquires the transformation matrices Rx, Ry, and Rz based on the coefficients A, B, and C and the data Data (Pa) of the point Pa. Then, the transformation matrix acquisition unit 1513 outputs the data on the acquired transformation matrices Rx, Ry, and Rz to the transformation matrix synthesis unit 1514.

  The transformation matrix combining unit 1514 receives data on the transformation matrices Rx, Ry, and Rz output from the transformation matrix acquiring unit 1513, and acquires a combination matrix R from the transformation matrices Rx, Ry, and Rz. Then, transformation matrix combination section 1514 outputs the acquired data on combination matrix R to inverse transformation matrix acquisition section 1515 and conversion section 1516.

The inverse transformation matrix acquisition unit 1515 inputs data on the synthesis matrix R output from the conversion matrix synthesis unit 1514. The inverse transformation matrix acquisition unit 1515 acquires the inverse transformation matrix R- ¹ from the synthesis matrix R, and outputs data on the acquired inverse transformation matrix R- ¹ to the viewpoint candidate point calculation unit 1519.

  The conversion unit 1516 inputs data on the synthesis matrix R output from the conversion matrix synthesis unit 1514 and data Data (Pb) of the point Pb output from the intersection calculation unit 1511. The conversion unit 1516 acquires data (coordinate data) for the point Pb 'as Data (Pb') from the combined matrix R and the data Data (Pb) of the point Pb. Then, conversion section 1516 outputs the obtained data Data (Pb ′) to coefficient acquisition section 1517 and viewpoint candidate point calculation section 1519.

  The coefficient acquisition unit 1517 inputs data Data (Pb ′) output from the conversion unit 1516 and data on the angle θ output from the angle acquisition unit 1518. The coefficient acquisition unit 1517 acquires the coefficient A ′ based on the data Data (Pb ′) and the angle θ. Then, the coefficient acquiring unit 1517 outputs data on the acquired coefficient A ′ to the viewpoint candidate point calculating unit 1519.

  The angle acquisition unit 1518 receives the signal Sig1 output from the first interface 6, and acquires the angle θ based on the signal Sig1. Then, the angle acquisition unit 1518 outputs the data on the acquired angle θ to the coefficient acquisition unit 1517 and the viewpoint candidate point calculation unit 1519.

The viewpoint candidate point calculation unit 1519 inputs data Data (Pc) output from the intersection calculation unit 1511 and data on the angle θ output from the angle acquisition unit 1518. In addition, viewpoint candidate point calculation section 1519 outputs data on coefficient A ′ output from coefficient acquisition section 1517, data Data (Pb ′) output from conversion section 1516, and output data from inverse transformation matrix acquisition section 1515. And data on the inverse transformation matrix R ^-1 .

The viewpoint candidate point calculation unit 1519 calculates the coordinate data Vp_cand of the viewpoint candidate point based on the data Data (Pc), the angle θ, the coefficient A ′, the data Data (Pb ′), and the inverse transformation matrix R− ^1. To get. Then, the viewpoint candidate point calculation unit 1519 outputs the acquired coordinate data Vp_cand of the viewpoint candidate point to the projection image acquisition unit 152.

  The projection image acquisition unit 152 includes an image D1 (test image Img1 (Img1_T3)) output from the first switch SW1 and coordinate data of a viewpoint candidate point output from the viewpoint candidate point calculation unit 1519 of the viewpoint candidate acquisition unit 151. Vp_cand is input. Further, the projection image acquisition unit 152 inputs the three-dimensional shape data 3D_data output from the three-dimensional shape data storage unit 5 and information P_prj on the projection point of the projection unit 3. The projection image acquisition unit 152 converts the image D1 (test image Img1 (Img1_T3)) into a viewpoint candidate specified by the coordinate data Vp_cand based on the three-dimensional shape data 3D_data and the information P_prj about the projection points of the projection unit 3. The image is converted into an image in which no geometric distortion occurs when viewed from a point, and the converted image is output to the fourth selector SEL4 as an image D41 (image signal D41).

  The fourth selector SEL4 inputs the image D4 output from the third selector SEL3, the image D41 output from the viewpoint specification processing unit 15, and the selection signal sel4. The fourth selector SEL4 selects one of the image D4 and the image D41 according to the selection signal sel4, and outputs the selected image to the fifth selector SEL5 as an image D5 (image signal D5). The selection signal sel4 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The fifth selector SEL5 receives the image D5 output from the fourth selector SEL4, the test image Img0 for three-dimensional shape measurement output from the test image storage unit 2, and the selection signal sel5. The fifth selector SEL5 selects one of the image D5 and the test image Img0 according to the selection signal sel5, and outputs the selected image to the projection unit 3 as an image Dout (image signal Dout). The selection signal sel5 is a control signal generated by a control unit that controls each functional unit of the projector device 100.

  The test image storage unit 2 stores a test image, and outputs a test image to the projection image adjustment unit 1 at a predetermined timing according to a request from the projection image adjustment unit 1.

  The projection unit 3 has an optical system for projecting an image. The projection unit 3 receives an image Dout (image signal Dout) output from the fifth selector SEL5 of the projection image adjustment unit 1, and projects the input image Dout (image signal Dout) on a projection target in a three-dimensional space. .

  The three-dimensional shape measurement unit 4 acquires three-dimensional measurement data of the projection target in the three-dimensional space, and outputs the acquired three-dimensional measurement data of the projection target to the three-dimensional shape data storage unit 5. The three-dimensional shape measurement unit 4 has, for example, a camera, captures a test image for three-dimensional shape measurement projected from the projection unit 3, and based on the captured image of the three-dimensional shape measurement test image. , To acquire three-dimensional measurement data of the projection target.

  The three-dimensional shape measurement unit 4 may, for example, (1) obtain a normal line of the projection plane and clarify the positional relationship between the projection unit 3 and the projection plane, or (2) project a projection axis (projection unit The angle between the optical axis of the optical system 3 and the projection plane is obtained, and the positional relationship between the projection unit 3 and the projection plane is clarified to acquire three-dimensional measurement data of the projection target (projection plane).

  The three-dimensional shape measuring unit 4 acquires three-dimensional measurement data of a projection target by acquiring a distance image by a TOF (Time Of Flight) method (for example, a phase difference method of the TOF method). Is also good. In this case, the three-dimensional shape measuring unit 4 has, for example, a light source for irradiating infrared rays and an image sensor for infrared rays. The reflected light of the infrared rays radiated from the light source is received by the image sensor and irradiated. The distance image is obtained by measuring the time until the infrared light is reflected from the projection target and returns. Then, the three-dimensional shape measuring unit 4 acquires three-dimensional measurement data of the projection target from the acquired distance image.

  The three-dimensional shape measuring unit 4 has a laser light source and a laser sensor, and obtains three-dimensional measurement data of the projection target by measuring the distance of the projection target from the flight time of the laser light. There may be. In this case, the three-dimensional shape measuring unit 4 irradiates the projection target with the laser light while sequentially changing the irradiation direction (while scanning the projection target with the laser light), and the laser light is reflected on the projection target and returns. By measuring the time until, the distance of the projection target is measured, and three-dimensional measurement data of the projection target is obtained.

  The three-dimensional shape data storage unit 5 receives the three-dimensional measurement data of the projection target acquired by the three-dimensional shape measurement unit 4 and stores the input three-dimensional measurement data of the projection target. The three-dimensional shape data storage unit 5 stores the three-dimensional measurement data in accordance with a request from the correction unit 11, the first rotation processing unit 13, and the viewpoint specification processing unit 15, by using the correction unit 11, the first rotation processing unit 13, and the viewpoint specification processing. Output to the unit 15.

  The first interface 6 is an interface between the projector device 100 and the controller 200. For example, a signal output from the controller 200 can be input to the projector device 100 via the first interface 6.

  The controller 200 includes a second interface 21, a control unit 22, and a user interface 23, as shown in FIG.

  The second interface 21 is an interface between the controller 200 and the projector device 100.

  The control unit 22 is a control unit that controls each functional unit of the controller 200.

  The user interface 23 generates a signal according to a user operation. For example, when the user presses a predetermined button provided on the controller, the user interface 23 generates a signal indicating that the button has been pressed.

<1.2: Operation of projection system>
The operation of the projection system 1000 configured as described above will be described below.

  In the following, processing performed by the projection system 1000 includes (1) three-dimensional shape measurement processing, (2) correction processing, (3) first rotation processing, (4) second rotation processing, and (5) viewpoint. The description will be given separately for the specific processing.

  For convenience of explanation, the operation of the projection system 1000 will be described with the projection target being the three-dimensional space shown in FIG.

  FIG. 5 is a diagram schematically showing a three-dimensional space, and shows a ceiling CL (inclined ceiling CL) to be projected, the projector device 100, and a user Usr on the floor FL. In FIG. 5, a-axis, b-axis, and c-axis are set as shown in FIG.

In the three-dimensional space shown in FIG. 5, the inclined ceiling CL is a plane, and is inclined with respect to the floor FL (floor plane FL), which is a plane, by an angle α shown in FIG. In FIG. 5, the projection point of the projection unit 3 of the projector device 100 is shown as a point P_prj, and the measurement point of the three-dimensional shape measurement unit 4 of the projector device 100 is shown as a point P_msr. In FIG. 5, the intersection of the optical axis (projection axis) Ax1 of the optical system of the projection unit 3 and the inclined ceiling CL is set as a point Pc, and the viewpoint position of the user (the center point of both eyes of the user) is set as a point Vp. Is shown.
Note that, as described above, the inclined ceiling CL is not limited to the case where the inclined ceiling CL is a plane that is rotated about the rotation axis as the b-axis with respect to the plane (ab plane) parallel to the floor FL. For example, the sloping ceiling CL rotates with respect to a plane (ab plane) parallel to the floor FL by rotating the rotation axis by the angle α using the rotation axis as the b axis, and furthermore, coincides with the plane rotated by the angle β using the rotation axis as the a axis. It may be a flat surface.

  For convenience of description, it is assumed that the center point of the user's left eye (left-eye viewpoint) and the center point of the right eye (right-eye viewpoint) are on a straight line parallel to the a-axis. The distance dL from the center point of the user's left eye (left-eye viewpoint) to the point Pc is equal to the distance dR (= dL) from the center point of the user's right eye (right-eye viewpoint) to the point Pc. Shall be.

  As shown in FIG. 5, an x-axis, a y-axis, and a z-axis are set. That is, the x-axis and the y-axis are set so that the optical axis Ax1 of the optical system of the projection unit 3 and the z-axis coincide with each other, and the normal of the xy plane is parallel to the z-axis. Then, as shown in FIG. 5, it is assumed that the origin Org of the xyz coordinates exists on the optical axis Ax1.

  In FIG. 5, a normal line Nm1 of the inclined ceiling plane CL passing through the point Pc and a point P1 on the normal line Nm1 are shown.

(1.2.1: Three-dimensional shape measurement processing)
First, the three-dimensional shape measurement processing will be described.

  In the following, in the projection system 1000, the three-dimensional shape measurement unit 4 has a camera, captures a three-dimensional shape measurement test image projected from the projection unit 3, and outputs the three-dimensional shape measurement test image. A case in which three-dimensional measurement data of a projection target is acquired based on the captured image described above will be described.

  The test image Img0 for three-dimensional shape measurement is output from the test image storage unit 2 to the fifth selector SEL5 of the projection image adjustment unit 1. The control unit generates a selection signal sel5 for selecting the terminal 0 shown in FIG. 2 of the fifth selector SEL5, and outputs it to the fifth selector SEL5. As a result, the test image Img0 for measuring the three-dimensional shape is output from the fifth selector SEL5 to the projection unit 3 as the image signal Dout.

  The projection unit 3 sets the test image Img0 for three-dimensional shape measurement to (A) the projection axis (the optical axis of the optical system of the projection unit 3) as the axis Ax1 shown in FIG. 5, and (B) sets the angle of view to FIG. And the projection point is projected onto the inclined ceiling plane CL to be projected as the point P_prj shown in FIG.

  The three-dimensional shape measurement unit 4 captures the three-dimensional shape measurement test image Img0 projected by the projection unit 3 using the three-dimensional shape measurement camera whose imaging point is the point P_msr. Then, the three-dimensional shape measurement unit 4 acquires the three-dimensional measurement data of the projection target by comparing the captured image with the test image Img0 for three-dimensional shape measurement.

  In the projection system 1000, the coordinates of the projection point P_prj in the three-dimensional space are known, the coordinates of the imaging point P_msr of the three-dimensional shape measurement unit 4 in the three-dimensional space are known, and the test image to be projected for the three-dimensional shape measurement Since Img0 is known, the coordinates of the projection target in the three-dimensional space can be calculated from the image captured at the imaging point P_msr. That is, by examining which pixel of the image captured at the imaging point P_msr corresponds to each pixel of the test image Img0 for three-dimensional shape measurement, the three-dimensional spatial position (light corresponding to each pixel) corresponding to each pixel is determined. (A three-dimensional space position where the light is reflected) can be specified. Therefore, by specifying the three-dimensional space position corresponding to each pixel, the coordinates of the three-dimensional space to be projected can be calculated.

  Note that the test image Img0 for measuring the three-dimensional shape may be, for example, an image formed by a sine wave signal having a predetermined cycle. In this case, for example, a plurality of images formed by a sine wave signal whose cycle and phase are shifted by a predetermined amount over a plurality of times are projected from the projection unit 3 onto a projection target, and a plurality of projected images are used to perform three-dimensional The shape measuring unit 4 may acquire the three-dimensional shape data of the projection target.

  The three-dimensional shape data of the projection target acquired by the three-dimensional shape measurement unit 4 by the above processing is output from the three-dimensional shape measurement unit 4 to the three-dimensional shape data storage unit 5 and stored in the three-dimensional shape data storage unit 5 Is done.

(1.2.2: Correction processing)
Next, the correction processing will be described.

  The test image Img1 is output from the test image storage unit 2 to the first selector SEL1. Then, the control unit generates a selection signal sel1 for selecting the terminal 0 shown in FIG. 2 of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1. As a result, the test image Img1 is output from the first selector SEL1 to the first switch SW1 as the image signal D1.

  FIG. 6 shows a first test image Img1_T1, which is an example of the test image Img1. As shown in FIG. 6, the first test image Img1_T1 is a rectangular image. The first test image Img1_T1 has a vertical center line L1v extending in the vertical direction (vertical direction) at the center in the horizontal direction (horizontal direction) of the image, as shown in FIG. It is an image of a pattern having a line L2h drawn in the (lateral direction).

  In the first test image Img1_T1 illustrated in FIG. 6, the portion where the intersection of the line L1v and the line L2h exists is the upper part of the image. In the projection system 1000, since it is necessary for the user to recognize the upper side of the image of the first test image Img1_T1, the test image used in the projection system 1000 has the upper part of the image like the first test image Img1_T1 shown in FIG. It is preferable to make the image easy to recognize. Note that the first test image Img1_T1 illustrated in FIG. 6 is an example, and any other image may be used as long as the image can recognize the upper part of the image.

  For convenience of explanation, a case will be described below where the first test image Img1_T1 shown in FIG. 6 is used as the test image.

  The correction unit 11 reads out the three-dimensional shape data 3D_data from the three-dimensional shape data storage unit 5 and acquires three-dimensional coordinate data of the inclined ceiling plane CL to be projected. Then, the correction unit 11 acquires the image so as to be an image without geometric distortion when viewed from the point P1, that is, the point P1 on the normal to the intersection Pc between the projection axis Ax1 and the ceiling plane CL. A correction process is performed on the image D1 using the three-dimensional coordinate data of the inclined ceiling plane CL thus obtained. Then, the correction unit 11 outputs the image after the correction processing to the second selector SEL2 as the image D2.

  The control unit generates a selection signal sel2 for selecting the terminal 0 shown in FIG. 2 of the second selector SEL2, and outputs the signal to the second selector SEL2. Accordingly, the image D2 is output from the second selector SEL2 to the third selector SEL3 as the image D3.

  The control unit generates a selection signal sel3 for selecting the terminal 0 shown in FIG. 2 of the third selector SEL3, and outputs the signal to the third selector SEL3. Thus, the image D3 is output from the third selector SEL3 to the fourth selector SEL4 as the image D4.

  The control unit generates a selection signal sel4 for selecting the terminal 1 shown in FIG. 2 of the fourth selector SEL4, and outputs the signal to the fourth selector SEL4. Accordingly, the image D4 is output from the fourth selector SEL4 to the fifth selector SEL5 as the image D5.

  The control unit generates a selection signal sel5 for selecting the terminal 1 shown in FIG. 2 of the fifth selector SEL5, and outputs the signal to the fifth selector SEL5. As a result, the image D5 is output from the fifth selector SEL5 to the projection unit 3 as the image Dout.

  The projection unit 3 projects the image Dout from the fifth selector SEL5 onto the inclined ceiling plane CL using the projection axis Ax1 shown in FIG. This state is referred to as “state 1”.

  FIG. 7 is a diagram schematically illustrating the three-dimensional space in the state 1. Specifically, FIG. 7 is a diagram schematically illustrating a state where the first test image Img1_T1 illustrated in FIG. 6 is projected on the inclined ceiling plane CL. In FIG. 7, in state 1, the first test image Img1_T1 viewed from the user's viewpoint Vp is shown as an image Img1_T1 (Vp). In FIG. 7, in state 1, the first test image Img1_T1 viewed from the point P1 is shown as an image Img1_T1 (P1).

  From FIG. 7, in state 1, the first test image Img1_T1 (Vp) viewed from the user's viewpoint Vp has a geometric distortion, but the first test image Img1_T1 (P1) viewed from the point P1. Shows that no geometric distortion has occurred (geometric distortion has hardly occurred).

  In the projection system 1000, after the correction processing is performed as described above, the first rotation processing is performed.

(1.2.3: First rotation process)
Next, the first rotation processing will be described.

  In the state 1 illustrated in FIG. 7, the user operates the controller 200 to display the image Img1_T1 projected on the inclined ceiling plane CL in the inclined ceiling plane CL around the point Pc in FIG. The image Img1_T1 is rotated in the direction indicated by Dir1 so that the vertical center line L1v of the image Img1_T1 looks like a vertical straight line when viewed from the viewpoint Vp of the user. That is, the user operates the controller 200 so that the plane including the vertical center line L1v of the image Img1_T1, the point Pc, and the viewpoint Vp is orthogonal to the straight line connecting both eyes of the user, and the inclined ceiling plane CL Is rotated about the point Pc in the inclined ceiling plane CL in the direction indicated by the arrow Dir1 in FIG. Note that the user rotates the image Img1_T1 such that the upper part of the image Img1_T1 (the part where the line L1v and the line L2h of the image Img1_T1 intersect) is the upper part when viewed from the user.

  This first rotation processing will be specifically described.

  As in the state 1, the test image Img1_T1 is output from the test image storage unit 2 to the first selector SEL1. Then, the control unit generates a selection signal sel1 for selecting the terminal 0 shown in FIG. 2 of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1. As a result, the test image Img1_T1 is output from the first selector SEL1 to the first switch SW1 as the image signal D1.

  The control unit generates a switching signal sw1 for selecting the terminal 1 shown in FIG. 2 of the first switch SW1, and outputs the signal to the first switch SW1. Accordingly, the test image Img1_T1 is output from the first switch SW1 to the correction unit 11 as the image signal D1.

  The correction unit 11 performs the same correction processing as that performed in the state 1 on the image D1. Then, the correction unit 11 outputs the image after the correction processing to the first rotation processing unit 13 as an image D2.

  The user operates the controller 200 to cause the projector device 100 to execute the first rotation process. For example, the controller 200 has two buttons for performing the first rotation process, and pressing the first button (one button) once causes the test image Img1_T1 to be displayed in the inclined ceiling plane CL. By executing a process of rotating the direction Dir1 to the left by a predetermined angle in FIG. 7 and pressing the second button (the other button) once, the test image Img1_T1 is displayed in the inclined ceiling plane CL. A process of rotating by a predetermined angle to the right in the direction Dir1 of No. 7 can be executed.

  Here, a case where the first button is pressed will be described.

  When the first button of the controller 200 is pressed once, the user interface 23 generates a signal indicating that the first button is pressed once. The control unit 22 of the controller 200 transmits a signal indicating that the first button has been pressed once to the first interface of the projector device 100 via the second interface 21 based on the signal generated by the user interface 23. Send.

  Note that the controller 200 can be held and operated by a user, and the controller 200 and the projector device 100 transmit and receive signals by wireless communication, for example.

  For example, in the state shown in FIG. 7, the user operates the controller 200 while holding the controller 200 in his hand and viewing the image Img1_T1 projected on the inclined ceiling plane CL, and the user is projected on the inclined ceiling plane CL. The image Img1_T1 is rotated.

  The first interface of the projector device 100 receives a signal transmitted from the second interface of the controller 200 and outputs the received signal to the second switch 12 as a signal Sig1.

  The second switch 12 selects the terminal A shown in FIG. 2 based on the switching signal sw2 generated by the control unit, and outputs the signal Sig1 to the first rotation processing unit 13.

The first rotation processing unit 13 performs a first rotation processing on the image D2 output from the correction unit 11 based on the signal Sig1, the three-dimensional shape data 3D_data, and information P_prj on the projection point of the projection unit 3. Execute
Note that the first rotation processing unit 13 uses the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3 without using the image D2 output from the correction unit 11 based on the signal Sig1. On the other hand, the first rotation process may be executed. In this case, the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3 need not be input to the first rotation processing unit 13.

  Specifically, when viewing the inclined ceiling plane CL from the point P1, the first rotation processing unit 13 displays the image Img1_T1 around the point Pc in the inclined ceiling plane CL as indicated by an arrow Dir1 shown in FIG. A conversion process (first rotation process) is performed on the image D2 so that the image is rotated leftward by a predetermined angle determined by the signal Sig1. Then, the first rotation processing unit 13 outputs the processed image to the second selector SEL2 as an image D21 (image signal D21).

  The second selector SEL2 selects the terminal 1 shown in FIG. 2 based on the selection signal sel2 generated by the control unit, and outputs the image D21 as the image D3 to the third selector SEL3.

  The third selector SEL3 selects the terminal 0 shown in FIG. 2 based on the selection signal sel3 generated by the control unit, and outputs the image D3 (= image D21) to the fourth selector SEL4 as the image D4.

  The fourth selector SEL4 selects the terminal 1 shown in FIG. 2 based on the selection signal sel4 generated by the control unit, and outputs the image D4 (= image D21) to the fifth selector SEL5 as the image D5.

  The fifth selector SEL5 selects the terminal 1 shown in FIG. 2 based on the selection signal sel5 generated by the control unit, and outputs the image D5 (= image D21) to the projection unit 3 as an image Dout.

  The projection unit 3 projects the image Dout (= image D21) from the fifth selector SEL5 onto the inclined ceiling plane CL with the projection axis Ax1 shown in FIG. Thus, when the inclined ceiling plane CL is viewed from the point P1, the image Img1_T1 is an image rotated about the point Pc by a predetermined angle in the direction Dir1.

  By repeating the above operation, the vertical center line L1v of the image Img1_T1 is made to appear as a vertical straight line when viewed from the user's viewpoint Vp. This state, that is, a state in which a plane including the vertical center line L1v of the image Img1_T1, the point Pc, and the viewpoint Vp is orthogonal to a straight line connecting both eyes of the user is referred to as “state 2”.

  FIG. 8 is a diagram schematically illustrating the three-dimensional space in the state 2. In FIG. 8, in state 2, the test image Img1_T1 viewed from the user's viewpoint Vp is shown as an image Img1_T1 (Vp).

  From FIG. 8, it can be seen that in state 2, the vertical center line L1v of the test image Img1_T1 (Vp) looks like a vertical straight line from the viewpoint Vp of the user.

  Then, in state 2, the first rotation processing unit 13 acquires and holds data for specifying a straight line including the vertical center line L1v. As data for specifying a straight line including the vertical center line L1v, for example, (1) data of an equation (linear equation) defining the straight line or (2) coordinates of two different points included in the straight line are specified. There is data etc. for

  Then, the first rotation processing unit 13 outputs a detection signal including data for specifying a straight line including the vertical center line L1v to the viewpoint specifying processing unit 15 as Det1.

  In the projection system 1000, after the first rotation processing is executed as described above, the second rotation processing is executed.

(1.2.4: Correction processing (for second test image))
FIG. 9 is a diagram illustrating a second test image Img1_T2, which is an example of the test image Img1 used for the second rotation processing.

  FIG. 10 is a diagram for explaining the second rotation process, and is a diagram (front view) of a three-dimensional space including the inclined ceiling plane CL viewed from the positive direction of the b-axis toward the negative direction.

  The correction unit 11 performs a correction process in the same manner as described above, using the input image as the second test image Img1_T2. That is, the following processing is executed.

  From the test image storage unit 2, the second test image Img1_T2 is output to the first selector SEL1. Then, the control unit generates a selection signal sel1 for selecting the terminal 0 shown in FIG. 2 of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1. As a result, the test image Img1_T2 is output from the first selector SEL1 to the first switch SW1 as the image signal D1.

  As shown in FIG. 9, the second test image Img1_T2 is a rectangular image. The second test image Img1_T2 is a pattern image having a horizontal center line L1h extending in the horizontal direction (horizontal direction) at the center in the vertical direction (vertical direction) of the image, as shown in FIG.

  Note that the second test image Img1_T2 illustrated in FIG. 9 is an example, and is not limited to the image illustrated in FIG. 9. The second test image Img1_T2 may be another image that can recognize horizontal distortion of the image. Good.

  For convenience of explanation, a case where the second test image Img1_T2 shown in FIG. 9 is used as a test image will be described below.

  The correction unit 11 reads out the three-dimensional shape data 3D_data from the three-dimensional shape data storage unit 5 and acquires three-dimensional coordinate data of the inclined ceiling plane CL to be projected. Then, the correction unit 11 acquires the image so as to be an image without geometric distortion when viewed from the point P1, that is, the point P1 on the normal to the intersection Pc between the projection axis Ax1 and the ceiling plane CL. A correction process is performed on the image D1 using the three-dimensional coordinate data of the inclined ceiling plane CL thus obtained. Then, the correction unit 11 outputs the image after the correction processing to the second selector SEL2 as the image D2.

  The control unit generates a selection signal sel2 for selecting the terminal 0 shown in FIG. 2 of the second selector SEL2, and outputs the signal to the second selector SEL2. Accordingly, the image D2 is output from the second selector SEL2 to the third selector SEL3 as the image D3.

  The control unit generates a selection signal sel3 for selecting the terminal 0 shown in FIG. 2 of the third selector SEL3, and outputs the signal to the third selector SEL3. Thus, the image D3 is output from the third selector SEL3 to the fourth selector SEL4 as the image D4.

  The control unit generates a selection signal sel4 for selecting the terminal 1 shown in FIG. 2 of the fourth selector SEL4, and outputs the signal to the fourth selector SEL4. Accordingly, the image D4 is output from the fourth selector SEL4 to the fifth selector SEL5 as the image D5.

  The control unit generates a selection signal sel5 for selecting the terminal 1 shown in FIG. 2 of the fifth selector SEL5, and outputs the signal to the fifth selector SEL5. As a result, the image D5 is output from the fifth selector SEL5 to the projection unit 3 as the image Dout.

  The projection unit 3 projects the image Dout from the fifth selector SEL5 onto the inclined ceiling plane CL using the projection axis Ax1 shown in FIG. This state is referred to as “state 3”.

  FIG. 10 is a diagram schematically illustrating the three-dimensional space in the state 3. Specifically, FIG. 10 is a diagram schematically illustrating a state where the second test image Img1_T2 illustrated in FIG. 9 is projected on the inclined ceiling plane CL. In FIG. 10, in state 3, the second test image Img1_T2 viewed from the user's viewpoint Vp is shown as an image Img1_T2 (Vp). In FIG. 10, in state 1, the second test image Img1_T2 viewed from the point P1 is shown as an image Img1_T2 (P1).

  From FIG. 10, in state 3, the second test image Img1_T2 (Vp) viewed from the user's viewpoint Vp has a geometric distortion, but the second test image Img1_T2 (P1) viewed from the point P1. Shows that no geometric distortion has occurred (geometric distortion has hardly occurred).

  In the projection system 1000, after the correction processing is performed as described above, the second rotation processing is performed.

(1.2.5: Second rotation process)
Next, the second rotation processing will be described.

  In the state 3 shown in FIG. 10, the user operates the controller 200 to rotate the image Img1_T2 projected on the inclined ceiling plane CL about the point Pc in the direction indicated by the arrow Dir2 in FIG. , The horizontal center line L1h of the image Img1_T2 is viewed as a horizontal straight line when viewed from the user's viewpoint Vp.

  This second rotation processing will be specifically described.

  From the test image storage unit 2, the second test image Img1_T2 is output to the first selector SEL1. Then, the control unit generates a selection signal sel1 for selecting the terminal 0 shown in FIG. 2 of the first selector SEL1, and outputs the selection signal sel1 to the first selector SEL1. As a result, the second test image Img1_T2 is output from the first selector SEL1 to the first switch SW1 as the image signal D1.

  The control unit generates a switching signal sw1 for selecting the terminal 1 shown in FIG. 2 of the first switch SW1, and outputs the signal to the first switch SW1. Thereby, the second test image Img1_T2 is output from the first switch SW1 to the correction unit 11 as the image signal D1.

  The correction unit 11 performs the same correction processing as that performed in the state 3 on the image D1. Then, the correction unit 11 outputs the image after the correction processing to the second selector SEL2 as the image D2.

  The control unit generates a selection signal sel2 for selecting the terminal 0 shown in FIG. 2 of the second selector SEL2, and outputs the signal to the second selector SEL2. Accordingly, the second test image Img1_T2 is output from the second selector SEL2 to the second rotation processing unit 14 as the image signal D3.

  The user operates the controller 200 to cause the projector device 100 to execute the second rotation process. For example, the controller 200 has two buttons for performing the second rotation processing, and pressing the first button (one of the buttons) once causes the second test image Img1_T2 to be displayed on the inclined ceiling plane CL. In FIG. 10, a process of rotating a predetermined angle in the left direction of the direction Dir2 in FIG. 10 is performed, and the second button (the other button) is pressed once, so that the second test image Img1_T2 becomes the inclined ceiling plane CL. , A process of rotating by a predetermined angle to the right of the direction Dir2 in FIG. 10 can be executed.

  Here, a case where the first button is pressed will be described.

  When the first button of the controller 200 is pressed once, the user interface 23 generates a signal indicating that the first button is pressed once. The control unit 22 of the controller 200 transmits a signal indicating that the first button has been pressed once to the first interface of the projector device 100 via the second interface 21 based on the signal generated by the user interface 23. Send.

  Note that the controller 200 can be held and operated by a user, and the controller 200 and the projector device 100 transmit and receive signals by wireless communication, for example.

  For example, in the state shown in FIG. 10, the user operates the controller 200 while holding the controller 200 in his hand and viewing the image Img1_T2 projected on the inclined ceiling plane CL, and the user is projected on the inclined ceiling plane CL. The image Img1_T2 is rotated.

  The first interface of the projector device 100 receives a signal transmitted from the second interface of the controller 200 and outputs the received signal to the second switch 12 as a signal Sig1.

  The second switch 12 selects the terminal B shown in FIG. 2 based on the switching signal sw2 generated by the control unit, and outputs the signal Sig1 to the second rotation processing unit 14.

The second rotation processing unit 14 performs a second rotation on the image D3 output from the second selector SEL2 based on the signal Sig1, the three-dimensional shape data 3D_data, and information P_prj on the projection point of the projection unit 3. Execute the process.
Note that the second rotation processing unit 14 does not use the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3 but outputs the image D3 output from the correction unit 11 based on the signal Sig1. On the other hand, the second rotation process may be executed. In this case, the three-dimensional shape data 3D_data and the information P_prj on the projection point of the projection unit 3 need not be input to the second rotation processing unit 14.

  Specifically, when viewing the inclined ceiling plane CL from the point P1, the second rotation processing unit 14 displays the image Img1_T2 around the point Pc in the inclined ceiling plane CL, as indicated by the arrow Dir2 in FIG. A conversion process (second rotation process) is performed on the image D3 so that the image is rotated in the counterclockwise direction by a predetermined angle determined by the signal Sig1. Then, the second rotation processing unit 14 outputs the processed image to the third selector SEL3 as an image D31 (image signal D31).

  The third selector SEL3 selects the terminal 1 shown in FIG. 2 based on the selection signal sel3 generated by the control unit, and outputs the image D31 as the image D4 to the fourth selector SEL4.

  The fourth selector SEL4 selects the terminal 1 shown in FIG. 2 based on the selection signal sel4 generated by the control unit, and outputs the image D4 as the image D5 to the fifth selector SEL5.

  The fifth selector SEL5 selects the terminal 1 shown in FIG. 2 based on the selection signal sel5 generated by the control unit, and outputs an image D5 (= image D31) to the projection unit 3 as an image Dout.

  The projection unit 3 projects the image Dout (= image D31) from the fifth selector SEL5 onto the inclined ceiling plane CL with the projection axis Ax1 shown in FIG. Thus, when the inclined ceiling plane CL is viewed from the point P1, the image Img1_T2 is an image rotated about the point Pc by a predetermined angle in the direction Dir2.

  By repeating the above operation, the horizontal center line L1h of the image Img1_T2 is made to appear as a horizontal straight line when viewed from the user's viewpoint Vp. This state, that is, the state in which the horizontal center line L1h of the image Img1_T2 and the straight line connecting both eyes of the user are parallel is referred to as "state 4."

  FIG. 11 is a diagram schematically illustrating the three-dimensional space in the state 4.

  From FIG. 11, it can be seen that in state 4, the horizontal center line L1h of the test image Img1_T1 (Vp) looks like a horizontal straight line when viewed from the user's viewpoint Vp.

  Then, in the state 4, the second rotation processing unit 14 acquires and holds data for specifying a straight line including the horizontal center line L1h. As data for specifying a straight line including the horizontal center line L1h, for example, (1) data of an equation (linear equation) defining the straight line or (2) coordinates of two different points included in the straight line are specified. There is data etc. for

  Then, the second rotation processing unit 14 outputs a detection signal including data for specifying a straight line including the horizontal center line L1h to the viewpoint specification processing unit 15 as Det2.

  In the projection system 1000, after the second rotation processing is executed as described above, the viewpoint specifying processing is executed.

(1.2.6: viewpoint identification processing)
Next, the viewpoint specifying process will be described.

  FIG. 12 is a diagram for explaining the viewpoint specifying process. Specifically, FIG. 12 shows a test image Img1_T2 (Img1_T2 (Vp)) projected on the inclined ceiling plane CL in the state 4 and a test image Img1_T1 (Img1_T1 (Img1_T2 (Vp)) projected on the inclined ceiling plane CL in the state 2 FIG. 2 is a diagram in which Img1_T1 (Vp)) is superimposed.

  The state illustrated in FIG. 12 is referred to as “state 4A”.

  FIG. 12 illustrates a viewpoint candidate curve Vp_cand_crv indicating a viewpoint candidate point.

  The intersection calculation unit 1511 of the viewpoint candidate acquisition unit 151 acquires the data of the first straight line from the detection signal Det1 output from the first rotation processing unit 13, and obtains the data on the straight line including the vertical center line L1v in the state 4A (first The coordinate data Data (Pa) of the point Pa on the straight line) is obtained.

  In addition, the intersection calculation unit 1511 acquires data of the second straight line from the detection signal Det2 output from the second rotation processing unit 14, and obtains data on a straight line (on the second straight line) including the horizontal center line L1h in the state 4A. The coordinate data Data (Pb) of the point Pb is obtained.

  In addition, the intersection calculation unit 1511 obtains coordinate data Data (Pc) of the intersection Pc of the first straight line and the second straight line. For example, the intersection calculating unit 1511 obtains a straight line equation of the first straight line and a straight line equation of the second straight line, and calculates the first straight line equation and the straight line equation of the second straight line from the obtained straight line equation. The coordinate data Data (Pc) (= (cx, cy, cz)) of the intersection Pc of the straight line and the second straight line is obtained.

The intersection calculation unit 1511 acquires coordinate data of the points Pa, Pb, and Pc in the xyz space defined by the x-axis, the y-axis, and the z-axis illustrated in FIG. 5 as the following data. .
Data (Pa) = (ax, ay, az)
Data (Pb) = (bx, by, bz)
Data (Pc) = (cx, cy, cz)
The intersection calculation unit 1511 outputs the obtained data Data (Pa) of the point Pa and the obtained data Data (Pb) of the point Pb to the coefficient calculation unit 1512.

The coefficient calculation unit 1512 calculates the data Data (Pa) of the point Pa and the data Data (Pb) of the point Pb from
A = −ax × by + ay × bx
B = −ax × bz + az × bx
C = -az × by + ay × bz
Is performed, and coefficients A, B, and C are obtained.

  Then, the coefficient calculation unit 1512 outputs information (data) on the obtained coefficients A, B, and C to the transformation matrix acquisition unit 1513.

そして、変換行列取得部１５１３は、取得した変換行列Ｒｘ、Ｒｙ、Ｒｚについてのデータを、変換行列合成部１５１４に出力する。 The transformation matrix acquisition unit 1513 acquires the transformation matrices Rx, Ry, and Rz by executing a process corresponding to the following equation using the coefficients A, B, and C and the data Pa (Pa) of the point Pa. I do.

Then, the transformation matrix acquisition unit 1513 outputs the data on the acquired transformation matrices Rx, Ry, and Rz to the transformation matrix synthesis unit 1514.

The transformation matrix synthesizing unit 1514 calculates, from the transformation matrices Rx, Ry, Rz,
R = Rz ・ Ry ・ Rx
(Matrix combining process) to obtain a combined matrix R.

  Then, transformation matrix combination section 1514 outputs the acquired data on combination matrix R to inverse transformation matrix acquisition section 1515 and conversion section 1516.

The inverse transformation matrix acquisition unit 1515 acquires the inverse transformation matrix R- ¹ from the synthesis matrix R, and outputs data on the acquired inverse transformation matrix R- ¹ to the viewpoint candidate point calculation unit 1519.

そして、変換部１５１６は、取得した点Ｐｂ’についてのデータＤａｔａ（Ｐｂ’）（＝（ｂｘ’，ｂｙ’，ｂｚ’））を、係数取得部１５１７と、視点候補点算出部１５１９とに出力する。 The conversion unit 1516 obtains data (coordinate data) for the point Pb ′ by executing a process corresponding to the following equation using the composite matrix R and the data Data (Pb) of the point Pb.

Then, the conversion unit 1516 outputs the data Data (Pb ′) (= (bx ′, by ′, bz ′)) for the acquired point Pb ′ to the coefficient acquisition unit 1517 and the viewpoint candidate point calculation unit 1519. I do.

  The angle acquisition unit 1518 receives the signal Sig1 output from the first interface 6, and acquires the angle θ based on the signal Sig1. For example, when the signal Sig1 is input, the angle acquisition unit 1518 changes the angle θ by a predetermined angle from the angle set before the signal Sig is input, based on the signal value of the signal Sig1. Is changed as follows.

  Then, the angle acquisition unit 1518 outputs the data on the acquired angle θ to the coefficient acquisition unit 1517 and the viewpoint candidate point calculation unit 1519.

そして、係数取得部１５１７は、取得した係数Ａ’についてのデータを、視点候補点算出部１５１９に出力する。 The coefficient obtaining unit 1517 obtains a coefficient A ′ based on the data Data (Pb ′) and the angle θ by a process corresponding to the following equation.

Then, the coefficient acquiring unit 1517 outputs data on the acquired coefficient A ′ to the viewpoint candidate point calculating unit 1519.

（２）ｂｙ’≦０のとき（０＜θ＜π／２）

そして、視点候補点算出部１５１９は、上記処理により取得した視点候補点の座標データＶｐ＿ｃａｎｄ（＝（ｘ，ｙ，ｚ））を投影画像取得部１５２に出力する。 Based on the data Data (Pc), the angle θ, the coefficient A ′, the data Data (Pb ′), and the inverse transformation matrix R- ¹ , the viewpoint candidate point calculation unit 1519 performs a process corresponding to the following equation. Is executed, coordinate data Vp_cand (= (x, y, z)) of the viewpoint candidate point is obtained.
(1) When by '> 0 (−π / 2 <θ <0)

(2) When by '≦ 0 (0 <θ <π / 2)

Then, the viewpoint candidate point calculation unit 1519 outputs the coordinate data Vp_cand (= (x, y, z)) of the viewpoint candidate point acquired by the above processing to the projection image acquisition unit 152.

  As described above, in the projection system 1000, the viewpoint specifying process is executed.

  FIG. 13 shows a third test image Img1_T3 which is an example of the test image Img1. The third test image Img1_T3 is an image used for the viewpoint specification processing.

  As shown in FIG. 13, the third test image Img1_T3 is a rectangular image. The third test image Img1_T3 has a vertical center line L1va extending in the vertical direction (vertical direction) at the center in the horizontal direction (horizontal direction) of the image, as shown in FIG. This is an image of a pattern having a line L2ha drawn in the horizontal direction) and having a circle C1 at the center of the image. Further, the third test image Img1_T3 has a horizontal center line L1ha extending in the horizontal direction (horizontal direction) at the center in the vertical direction (vertical direction) of the image, as shown in FIG.

  In the third test image Img1_T3 illustrated in FIG. 13, a portion where the intersection between the line L1va and the line L2ha exists is the upper part of the image.

  Further, as shown in FIG. 13, the third test image Img1_T3 includes a first lattice pattern, which is a white square pattern, and a second lattice pattern, which is a black square pattern, in the horizontal direction on the image, and Has a lattice pattern formed by being alternately arranged in the vertical direction on the image.

  Note that the third test image Img1_T3 illustrated in FIG. 13 is an example, and the image used for the viewpoint specification processing is another image as long as the image can be recognized by the user when projected. You may.

  In the projection system 1000, when performing the viewpoint identification processing, the third test image Img1_T3 is output from the test image storage unit 2 to the first selector SEL1 of the projection image adjustment unit 1.

  Then, the terminal 0 is selected by the first selector, and the terminal 0 is selected by the first switch SW1, so that the third test image Img1_T3 is input to the viewpoint specification processing unit 15.

  The projection image acquisition unit 152 of the viewpoint specification processing unit 15 converts the input third test image Img1_T3 based on the three-dimensional shape data 3D_data and information P_prj on the projection point of the projection unit 3 into coordinate data Vp_cand (= When viewed from the viewpoint candidate point specified by (x, y, z)), the image is converted into an image in which geometric distortion does not occur. Then, the projection image acquisition unit 152 outputs the converted image to the fourth selector SEL4 as an image D41 (image signal D41).

  Then, the image D41 is output from the fifth selector SEL5 to the projection unit 3 as an image Dout.

  The projection unit 3 projects the image Dout (image D41) on the inclined ceiling CL.

  The user operates the controller 200 such that the degree of distortion of the third test image Img1_T3 projected on the inclined ceiling CL is reduced.

  In accordance with the operation of the controller 200 by the user, the projection system 1000 repeatedly executes the above-described viewpoint specifying processing.

  In FIG. 12, the viewpoint candidate points specified by the Nth (N: natural number) viewpoint specification process are indicated as Vp_cand (N), and the viewpoint candidate points specified by the (N + 1) th viewpoint specification process are represented by Vp_cand (N). N + 1). It is assumed that a viewpoint candidate point Vp_cand (N + 1) exists on a straight line connecting the point Pc and the user's viewpoint Vp.

  When the user operates the controller 200 to execute the (N + 1) th viewpoint specifying process, the viewpoint candidate point becomes Vp_cand (N + 1), and when viewed from the user's viewpoint Vp, an image without geometric distortion is obtained. It is projected on the inclined ceiling CL.

  FIG. 14 illustrates a case where the projection system 1000 performs the (N + 1) th viewpoint specifying process and specifies a viewpoint candidate point as Vp_cand (N + 1), and the third test image Img1_T3 is projected on the inclined ceiling CL. It is a figure which shows a state (this state is called "state 5").

As can be seen from FIG. 14, since the viewpoint candidate point Vp_cand (N + 1) exists on the straight line connecting the point Pc and the user's viewpoint Vp, when viewed from the user in state 5, the third test image Img1_T3 is Is recognized as an image without geometric distortion.

  As described above, in the projection system 1000, the viewpoint specifying process is executed, and an image without geometric distortion (an image with reduced geometric distortion) when viewed from the user's viewpoint Vp is converted into a tilted ceiling plane. CL can be projected.

  The user confirms that the geometric distortion of the image projected on the inclined ceiling plane CL is sufficiently reduced (state 5), and adjusts the image in the projector device 100 (to reduce the geometric distortion of the image). Adjustment process). For example, the user operates the controller 200 to end the adjustment processing (adjustment processing for reducing geometric distortion of an image) in the projector device 100.

  The projector device 100 receives from the controller 200 a signal for ending the adjustment process (adjustment process for reducing geometric distortion of an image) in the projector device 100.

  When receiving the above signal from the controller 200, the projector device 100 causes the image to be projected on the inclined ceiling CL according to the current setting.

  Then, when the mode is switched to the mode of displaying (projecting) the image signal (or video signal) Din, the projector device 100 causes the first selector SEL1 to select the terminal 1 and further sets the first switch In SW <b> 1, the image signal Din is output to the viewpoint specification processing unit 15 so that the terminal 0 is selected. Then, the viewpoint identification processing unit 15 converts the image Din in which no geometric distortion occurs when viewed from the user's viewpoint Vp (point Vp_cand (N + 1)) into an image, and sets the converted image as an image D41. 4 selector SEL4.

  The projector device 100 is configured to select the terminal 0 in the fourth selector SEL4 and the terminal 1 in the fifth selector, respectively. As a result, the image D41 is output to the projection unit 3 as the image Dout.

  Then, the image Dout (image D41) is projected onto the inclined ceiling CL by the projection unit 3 of the projector device 100.

  Thereby, the image (video) in which the image Din (or video Din) input to the projector device 100 is projected on the inclined ceiling plane CL has no geometric distortion when viewed from the user's viewpoint Vp ( (Reduced) image (video).

As described above, the projection system 1000 projects the test image showing the upper part when displayed on the inclined ceiling plane CL, and the user performs the following processes (1) to (3) on the projector device 100 by the controller 200. The execution can eliminate (reduce) the geometric distortion of the image projected on the inclined ceiling plane CL.
(1) A process of rotating the projection image in the inclined ceiling plane CL so that the vertical center line L1v of the first test image Img1_T1 looks vertical when viewed from the user (first rotation process).
(2) A process of rotating the projected image in the inclined ceiling plane CL such that the horizontal center line L1h of the second test image Img1_T2 looks horizontal when viewed from the user (second rotation process).
(3) After the completion of the first rotation processing, the first straight line including the vertical center line L1v of the first test image Img1_T1 projected on the inclined ceiling CL and the inclined ceiling CL after the completion of the second rotation processing. A viewpoint candidate point is specified based on a second straight line including the horizontal center line L1h of the projected second test image Img1_T2, and an image without geometric distortion when viewed from the specified viewpoint candidate point. Is generated, and the image is projected (viewpoint specifying process).

  Therefore, in the projection system 1000, it is possible to easily and appropriately eliminate (reduce) geometric distortion of an image projected on an inclined projection surface (for example, an inclined ceiling) without using a device having a photographing function. it can.

<1.3: Projection method>
Next, a projection method executed by the projection system 1000 will be described.

  FIG. 15 is a diagram illustrating a flowchart of the projection method executed by the projection system 1000.

  FIG. 16 is a diagram illustrating a flowchart of the viewpoint identification processing of the projection method executed by the projection system 1000.

  Hereinafter, the projection method executed by the projection system 1000 will be described with reference to the flowcharts of FIGS.

(Step S1):
In step S1, the projection system 1000 executes a first rotation process.

  The first rotation process is the same as the process described in “(1.2.3: First rotation process)”.

  In the first rotation process, for example, the first test image Img1_T1 illustrated in FIG. 6 is projected onto the inclined ceiling CL by the projector device 100.

  Then, the user operates the controller 200 until the vertical center line L1v looks like a vertical straight line in the first test image Img1_T1 projected on the inclined ceiling CL.

(Step S2):
In step S2, the projection system 1000 performs a second rotation process.

  The second rotation process is a process similar to the process described in “(1.2.5: Second rotation process)”.

  In the second rotation process, for example, the second test image Img1_T2 illustrated in FIG. 9 is projected on the inclined ceiling CL by the projector device 100.

  Then, the user operates the controller 200 until the horizontal center line L1h looks like a horizontal straight line in the second test image Img1_T2 projected on the inclined ceiling CL.

(Step S3):
In step S3, the projection system 1000 performs a viewpoint specifying process.

  The viewpoint specifying process is a process similar to the process described in “(1.2.6: viewpoint specifying process)”.

(Step S301):
In step S301, the intersection calculation unit 1511 of the viewpoint candidate acquisition unit 151 acquires the data of the first straight line from the detection signal Det1 output from the first rotation processing unit 13, and obtains the straight line including the vertical center line L1v in the state 4A. The coordinate data Data (Pa) (= (ax, ay, az)) of the upper (first straight line) point Pa is acquired.

  In addition, the intersection calculation unit 1511 acquires data of the second straight line from the detection signal Det2 output from the second rotation processing unit 14, and obtains data on a straight line (on the second straight line) including the horizontal center line L1h in the state 4A. The coordinate data Data (Pb) (= (bx, by, bz)) of the point Pb is acquired.

(Step S302):
In step S302, the intersection calculation unit 1511 acquires the coordinate data Data (Pc) (= (cx, cy, cz)) of the intersection Pc of the first straight line and the second straight line. For example, the intersection calculating unit 1511 obtains a straight line equation of the first straight line and a straight line equation of the second straight line, and calculates the first straight line equation and the straight line equation of the second straight line from the obtained straight line equation. The coordinate data Data (Pc) (= (cx, cy, cz)) of the intersection Pc of the straight line and the second straight line is obtained.

The intersection calculation unit 1511 acquires coordinate data of the points Pa, Pb, and Pc in the xyz space defined by the x-axis, the y-axis, and the z-axis illustrated in FIG. 5 as the following data. .
Data (Pa) = (ax, ay, az)
Data (Pb) = (bx, by, bz)
Data (Pc) = (cx, cy, cz)
(Step S303):
In step S303, the coefficient calculation unit 1512 calculates the data Data (Pa) of the point Pa and the data Data (Pb) of the point Pb from
A = −ax × by + ay × bx
B = −ax × bz + az × bx
C = -az × by + ay × bz
Is performed, and coefficients A, B, and C are obtained.

（ステップＳ３０５）：
ステップＳ３０５において、変換行列合成部１５１４は、変換行列Ｒｘ、Ｒｙ、Ｒｚから、
Ｒ＝Ｒｚ・Ｒｙ・Ｒｘ
に相当する処理（行列の合成処理）を行い、合成行列Ｒを取得する。 (Step S304):
In step S304, the conversion matrix acquisition unit 1513 performs a process corresponding to the following equation using the coefficients A, B, and C and the data Data (Pa) of the point Pa, thereby obtaining the conversion matrices Rx and Ry. , Rz.

(Step S305):
In step S305, the transformation matrix synthesizing unit 1514 uses the transformation matrices Rx, Ry, and Rz to calculate
R = Rz ・ Ry ・ Rx
(Matrix combining process) to obtain a combined matrix R.

(Step S306):
In step S306, the inverse transformation matrix acquisition unit 1515 acquires the inverse transformation matrix R- ¹ from the composite matrix R.

（ステップＳ３０８）：
ステップＳ３０８において、角度取得部１５１８は、第１インターフェース６から出力される信号Ｓｉｇ１を入力し、信号Ｓｉｇ１に基づいて、角度θを取得する。角度取得部１５１８は、例えば、信号Ｓｉｇ１が入力された場合、信号Ｓｉｇ１の信号値に基づいて、角度θが、信号Ｓｉｇが入力される前に設定されていた角度から、所定の角度だけ変化するように角度θを変化させる。 (Step S307):
In step S307, the conversion unit 1516 performs a process corresponding to the following equation using the composite matrix R and the data Data (Pb) of the point Pa, thereby obtaining data (coordinate data) for the point Pb ′. To get.

(Step S308):
In step S308, the angle acquisition unit 1518 receives the signal Sig1 output from the first interface 6, and acquires the angle θ based on the signal Sig1. For example, when the signal Sig1 is input, the angle acquisition unit 1518 changes the angle θ by a predetermined angle from the angle set before the signal Sig is input, based on the signal value of the signal Sig1. Is changed as follows.

（ステップＳ３０９）：
ステップＳ３０９において、視点候補点算出部１５１９は、データＤａｔａ（Ｐｃ）と、角度θと、係数Ａ’と、データＤａｔａ（Ｐｂ’）と、逆変換行列Ｒ^−１とに基づいて、以下の数式に相当する処理を実行することで、視点候補点の座標データＶｐ＿ｃａｎｄ（＝（ｘ，ｙ，ｚ））を取得する。
（１）ｂｙ’＞０のとき（−π／２＜θ＜０）

（２）ｂｙ’≦０のとき（０＜θ＜π／２）

（ステップＳ３１０）：
ステップＳ３１０において、投影画像取得部１５２は、３次元形状データ３Ｄ＿ｄａｔａと、投影部３の投影点についての情報Ｐ＿ｐｒｊとに基づいて、入力される第３テスト画像Ｉｍｇ１＿Ｔ３を、座標データＶｐ＿ｃａｎｄ（＝（ｘ，ｙ，ｚ））により特定される視点候補点から見たときに、幾何的な歪みが発生しない画像に変換する。 The coefficient obtaining unit 1517 obtains a coefficient A ′ based on the data Data (Pb ′) and the angle θ by a process corresponding to the following equation.

(Step S309):
In step S309, the viewpoint candidate point calculation unit 1519 calculates the following formula based on the data Data (Pc), the angle θ, the coefficient A ′, the data Data (Pb ′), and the inverse transformation matrix R ^−1. By executing the processing corresponding to the above, coordinate data Vp_cand (= (x, y, z)) of the viewpoint candidate point is obtained.
(1) When by '> 0 (−π / 2 <θ <0)

(2) When by '≦ 0 (0 <θ <π / 2)

(Step S310):
In step S310, the projection image acquisition unit 152 converts the input third test image Img1_T3 into coordinate data Vp_cand (= (x) based on the three-dimensional shape data 3D_data and the information P_prj about the projection points of the projection unit 3. , Y, z)), the image is converted into an image in which no geometric distortion occurs when viewed from the viewpoint candidate point specified by the viewpoint.

(Step S311):
In step S311, the projection unit 3 projects the image (image Dout (image D41)) acquired in step S310 on the inclined ceiling CL.

(Step S312):
In step S312, the user determines whether or not the third test image Img1_T3 projected on the inclined ceiling CL has a degree of distortion (sufficiently small).

  If it is determined that the degree of distortion of the third test image Img1_T3 projected on the inclined ceiling CL is not sufficiently small, the process returns to step S301. On the other hand, when it is determined that the degree of distortion of the third test image Img1_T3 projected on the inclined ceiling CL is sufficiently small, the processing ends (the viewpoint specifying processing is completed).

As described above, in the projection method executed by the projection system 1000, a test image that shows the upper part when displayed is projected on the inclined ceiling plane CL, and the user performs the following (1) to (3) using the controller 200. By causing the projector device 100 to execute the processing, it is possible to eliminate (reduce) the geometric distortion of the image projected on the inclined ceiling plane CL.
(1) A process of rotating the projection image in the inclined ceiling plane CL so that the vertical center line L1v of the first test image Img1_T1 looks vertical when viewed from the user (first rotation process).
(2) A process of rotating the projected image in the inclined ceiling plane CL such that the horizontal center line L1h of the second test image Img1_T2 looks horizontal when viewed from the user (second rotation process).
(3) After the completion of the first rotation processing, the first straight line including the vertical center line L1v of the first test image Img1_T1 projected on the inclined ceiling CL and the inclined ceiling CL after the completion of the second rotation processing. A viewpoint candidate point is specified based on a second straight line including the horizontal center line L1h of the projected second test image Img1_T1, and an image without geometric distortion when viewed from the specified viewpoint candidate point. Is generated, and the image is projected (viewpoint specifying process).

  Therefore, the projection method executed by the projection system 1000 easily and appropriately eliminates geometric distortion of an image projected on an inclined projection plane (for example, an inclined ceiling) without using an apparatus having a photographing function. (Reduce).

≫Principle of viewpoint identification processing≫
Here, the principle of the viewpoint identification processing will be described.

Three different points in a three-dimensional space (xyz coordinate space)
A0 = (ax0, ay0, az0)
B0 = (bx0, by0, bz0)
C0 = (cx0, cy0, cz0)
Let Px be a point at which the vector Vec (C0, A0) (vector from point C0 to point A0) and the vector Vec (C0, B0) (vector from point C0 to point B0) are orthogonal to each other.

  Identifying the point Px is a problem (proposition) to be solved.

To solve this problem, the following two processes may be performed.
(1) When viewed from a certain point P0 (for example, a certain point (0, 0, 1) on the z-axis), the vector Vec (C0, A0) and the vector Vec (C0, B0) appear to be orthogonal to each other. Find the geometric transformation that gives In addition, here, for convenience of explanation, the point P0 will be described as (0, 0, 1) below.
P0 = GmtrCnv (Px)
Px = (x, y, z)
P0 = (0,0,1)
GmtrCnv: geometric transformation (2) From the point at which point P0 is subjected to the inverse transformation of the geometric transformation obtained in (1), vector Vec (C0, A0) and vector Vec (C0, B0) are orthogonal to each other. You should see it. Therefore,
Px = GmtrCnv ^-1 (P0)
GmtrCnv ^-1 : A process corresponding to the inverse transformation of the geometric transformation GmtrCnv is performed to obtain a vector Vec (C0, A0) (vector from point C0 to point A0) and a vector Vec (C0, B0) (vector from point C0 to point (A vector to B0) can be obtained. In the following, a vector from the point m0 to the point m1 is referred to as Vec (m0, m1).

  FIG. 17 is a diagram for explaining the principle of the viewpoint identification processing.

The point A0, the point B0, and the point C0 may be any points as long as they are three different points. However, for convenience of explanation, a case where the point C0 is the origin (= (0, 0, 0)) will be described below. explain. That is, the case shown in FIG. 17B will be described below as an example. That is,
A0 = (ax, ay, az)
B0 = (bx, by, bz)
C0 = (0,0,0)
Is described.

  The rotation processing is performed so that the vector Vec (C0, A0) overlaps the y-axis and the vector Vec (C0, B0) is on the xy plane. This rotation processing is realized by rotation processing about the x axis, rotation processing about the y axis, and rotation processing about the z axis.

If the points after the rotation processing are point A0 ', point B0', and point C0 ', the coordinates of point A0', point B0 ', and point C0' can be expressed as follows.
A0 ′ = (ax ′, ay ′, az ′) = (0, ay ′, 0)
B0 '= (bx', by ', bz') = (bx ', by', 0)
The matrix for performing the rotation process can be expressed as follows.
R = Rx ・ Ry ・ Rz
Note that Rx is a matrix representing rotation processing about the x axis, Ry is a matrix representing rotation processing about the y axis, and Rz is a matrix representing rotation processing about the z axis. It is.

  FIG. 18 is a diagram showing points A0 ', B0', and C0 '(= point C0) after executing the above processing.

  Next, the vector Vec (C0 ′, A0 ′) is set on the yz plane, and the vector Vec (C0 ′, B0 ′) is set on the xz plane so that the vector is rotated about the y axis. Are sequentially executed.

なお、ρはｘ軸を中心とする回転処理の回転角度であり、θはｙ軸を中心とする回転処理の回転角度である。 Assuming that the points after the rotation processing are points A0 ″, B0 ″ and C0 ″, the coordinates of the points A0 ″, B0 ″ and C0 ″ are as follows.
A0 ″ = (ax ″, ay ″, az ″) = (0, ay ″, az ″)
B0 ″ = (bx ″, by ″, bz ″) = (bx ″, 0, bz ″)
FIG. 19 is a diagram illustrating points A0 ″, B0 ″, and C0 ″ (= point C0) after the above processing is performed.
Then, a rotation matrix R ′ for executing the rotation processing is as follows.

Here, ρ is the rotation angle of the rotation processing about the x axis, and θ is the rotation angle of the rotation processing about the y axis.

ある点Ｐ０（上記では、Ｐ０＝（０，０，１））に対して、上記幾何変換の逆変換を施した点（この点を点Ｖｐ（＝（ｘ，ｙ，ｚ））とする）から、ベクトルＶｅｃ（Ｃ０，Ａ０）と、ベクトルＶｅｃ（Ｃ０，Ｂ０）とが直交して見えるはずである。 Then, from the coordinate relationship before and after the rotation processing, the rotation matrix R ′ is obtained as follows.

A point obtained by subjecting a point P0 (in the above, P0 = (0, 0, 1)) to inverse transformation of the above geometric transformation (this point is referred to as a point Vp (= (x, y, z))) Therefore, the vector Vec (C0, A0) and the vector Vec (C0, B0) should appear orthogonal.

Ｖ’＝（０，０，１）
図１２（状態４Ａ）における点Ｐｃから点Ｐａまでのベクトルを、上記のベクトルＶｅｃ（Ｃ０，Ａ０）とし、図１２（状態４Ａ）における点Ｐｃから点Ｐｂまでのベクトルを、上記のベクトルＶｅｃ（Ｃ０，Ｂ０）とすることで、上記と同様にして、図１２（状態４Ａ）の点Ｐｃ、点Ｐａを通る直線（第１直線）と、点Ｐｃ、点Ｐｂを通る直線（第２直線）とが直交して見える点（視点候補点Ｖｐ）を算出することができる。 That is, the following formula is established.

V '= (0,0,1)
The vector from the point Pc to the point Pa in FIG. 12 (state 4A) is the above vector Vec (C0, A0), and the vector from the point Pc to the point Pb in FIG. 12 (state 4A) is the vector Vec ( By setting C0, B0), similarly to the above, a straight line (first straight line) passing through the points Pc and Pa in FIG. 12 (state 4A) and a straight line (second straight line) passing through the points Pc and Pb Can be calculated (a viewpoint candidate point Vp).

  The viewpoint specifying process is executed based on the above principle.

[Other embodiments]
In the embodiment, the first interface and / or the second interface may be an interface for wireless communication or an interface for wired communication.

  Further, in the above embodiment, the controller 200 has two buttons, a first button and a second button. When the user operates the first button or the second button, the first rotation processing, Although the case where the second rotation process and the trapezoidal correction process are executed has been described, the present invention is not limited to this. For example, the controller 200 has two buttons for the first rotation processing, has two buttons for the second rotation processing, and has two buttons for the viewpoint identification processing, You may have it separately. Further, the controller 200 may have an interface other than the buttons. For example, the controller 200 may have a joystick, and the user may operate the joystick to cause the projector device 100 to execute the first rotation processing, the second rotation processing, and the viewpoint identification processing. Alternatively, the controller 200 has a display panel with a touch panel function, displays an icon on the display panel, assigns an execution function of the first rotation processing, the second rotation processing, and the viewpoint identification processing to the icon, and The first rotation processing, the second rotation processing, and the trapezoid correction processing may be executed by the user operating the icon.

  In the above embodiment, the projection surface of the projection system has been described as being the ceiling of the inclined plane shown in FIG. 5 and the like. However, the present invention is not limited to this. (Or with respect to the user's viewpoint), it may be inclined in any direction and angle. Even in this case, by performing the same processing as above, an image without geometric distortion (an image with reduced geometric distortion) when viewed from the user's viewpoint with the inclined plane as the projection plane ) Can be projected.

  Words such as “horizontal”, “vertical”, “vertical”, “horizontal”, and “plane” are strictly meaning “horizontal”, “vertical”, “vertical”, “horizontal”, etc. The wording of the concept does not include only “substantially horizontal”, “substantially vertical”, “substantially vertical”, “substantially horizontal”, “substantially flat” and the like. Further, words such as “horizontal”, “vertical”, “vertical”, “horizontal”, and “plane” are words including a concept that allows a design error or a measurement error.

  In the projection system, the projector device, and the controller described in the above embodiment, each block may be individually formed into one chip by a semiconductor device such as an LSI, or may be formed into one chip so as to include a part or the whole. good.

  Here, the LSI is used, but it may be called an IC, a system LSI, a super LSI, or an ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  A part or all of the processing of each functional block (each functional unit) in each of the above embodiments may be realized by a program. Part or all of the processing of each functional block in each of the above embodiments is performed by a central processing unit (CPU) in a computer. Further, programs for performing the respective processes are stored in a storage device such as a hard disk and a ROM, and are executed by being read from the ROM or from the RAM. For example, a part or all of the processing of each functional block (each functional unit) of each of the above embodiments may be executed by a configuration as shown in FIG.

  Further, each process of the above embodiment may be realized by hardware, or may be realized by software (including a case where it is realized together with an OS (Operating System), middleware, or a predetermined library). Further, it may be realized by mixed processing of software and hardware.

  Further, the execution order of the processing method in the embodiment is not necessarily limited to the description of the embodiment, and the execution order can be changed without departing from the gist of the invention.

  In the adjustment processing (adjustment processing for reducing geometric distortion of an image) in the above embodiment, if the processing contents of the correction processing, the first rotation processing, the second rotation processing, and the viewpoint identification processing are determined, At the time of actual projection, it is not always necessary to separately execute the correction processing, the first rotation processing, the second rotation processing, and the viewpoint identification processing, and image processing for executing all conversions at once is performed. Is also good.

  A computer program that causes a computer to execute the above-described method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of the computer-readable recording medium include a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, large-capacity DVD, next-generation DVD, and semiconductor memory. .

  The computer program is not limited to the one recorded on the recording medium, and may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, or the like.

  In addition, the word “part” may be a concept including “circuitry”. The circuit may be realized in whole or in part by hardware, software, or a mixture of hardware and software.

  The specific configuration of the present invention is not limited to the above embodiment, and various changes and modifications can be made without departing from the spirit of the invention.

1000 Projection system 100 Projector device 1 Projected image adjustment unit 2 Test image storage unit 3 Projection unit 4 3D shape measurement unit 5 3D shape data storage unit 6 First interface

Claims (9)

A plane parallel to a plane including the left-eye viewpoint and the right-eye viewpoint, and a plane inclined with respect to the plane that is a common tangent plane of the right-eye cornea and the left-eye cornea is set as a projection plane, and the user's viewpoint A projection system for projecting an image so that when viewed from a geometric image distortion is reduced,
A projection unit for projecting an image on the projection plane, the projection unit projecting a first test image having a vertical reference line defining a vertical direction of the image, and a second test having a horizontal reference line defining a horizontal direction of the image. Said projection unit for projecting an image,
A three-dimensional shape measuring unit for measuring a three-dimensional shape of the projection plane;
A controller that generates a control signal based on a user operation;
(1A) on the basis of the three-dimensional shape three-dimensional shape data measured by the measuring unit, a point on the normal line of passing Ru said projection surface a projection center point which is the intersection between the projection plane and the projection axis of said projection portion when viewed from the first point is, so that such a state where the geometric image distortion is reduced, performs a correction process is an image conversion process on the first test image,
(2A) determining a first rotation angle in accordance with the control signal, and projecting the corrected first test image onto the projection surface by the projection unit when the projection surface is viewed from the first point; Performing a first rotation process as an image conversion process on the first test image after the correction process so that the image is an image rotated by the first rotation angle around the projection center point;
(1B) on the basis of the three-dimensional shape three-dimensional shape data measured by the measuring unit, a point on the normal line of passing Ru said projection surface a projection center point which is the intersection between the projection plane and the projection axis of said projection portion when viewed from the first point is, so that such a state where the geometric image distortion is reduced, performs a correction process is an image conversion process on the second test image,
(2B) When the second rotation angle is determined according to the control signal and the projection plane is viewed from the first point, projection of the second test image after the correction processing onto the projection plane by the projection unit. Performing a second rotation process as an image conversion process on the second test image after the correction process so that the image is an image rotated by the second rotation angle around the projection center point;
(3) a first straight line including the vertical reference line on the projection plane of the first test image after the first rotation process, and the projection of the second test image after the second rotation process; A viewpoint candidate point where the first straight line and the second straight line appear orthogonal to each other is determined based on a second straight line that is a straight line including the vertical reference line on a plane, and the determined viewpoint candidate point is determined. in so that such a state of geometrical image distortion is reduced when viewed from the perspective candidate points, to perform the viewpoint specific process of converting the image projected on the projection plane, a projection image adjustment unit,
Equipped with a,
The projection image adjustment unit,
A line on the projection image of the first test image corresponding to the vertical reference line of the first test image is a vertical straight line when viewed from a user's viewpoint by executing the first rotation process. After the first state is reached, the second rotation process is executed,
Further, the projection image adjustment unit includes:
By executing the second rotation process, a line on the projection image of the second test image corresponding to the horizontal reference line of the second test image is horizontal when viewed from a user's viewpoint. After the second state, which is a straight line state, the viewpoint specifying process is executed,
Further, the projection image adjustment unit includes:
According to the control signal from the controller, the determined viewpoint candidate point set before the current time, a point included in the viewpoint candidate point, other than the determined viewpoint candidate point set before the current time By changing to the point of, the determined viewpoint candidate point is changed, and using the changed determined viewpoint candidate point, the viewpoint specifying process is executed,
Projection system. The projection image adjustment unit,
In the xyz coordinate space in which the x-axis and the y-axis are set such that the optical axis of the projection unit is the z-axis and the plane whose normal is the optical axis of the projection unit is the xy plane,
A point on a first straight line that is a straight line including the vertical reference line on the projection plane of the first test image after the first rotation processing is a point Pa, and coordinates of the point Pa are (ax, ay, az). age,
A point on a second straight line including the horizontal reference line on the projection plane of the second test image after the second rotation processing is a point Pb, and coordinates of the point Pb are (bx, by, bz). age,
When the intersection of the first straight line and the second straight line is calculated as a point Pc, and the coordinates of the calculated point Pc are (cx, cy, cz),
A = −ax × by + ay × bx
B = −ax × bz + az × bx
C = -az × by + ay × bz
Is performed, and coefficients A, B, and C are obtained,

To obtain the rotation matrices Rx, Ry, Rz,
The composite matrix of the rotation matrices Rx, Ry and Rz is R
R = Rz ・ Ry ・ Rx
Is obtained by executing the process corresponding to
Obtaining the inverse of the rotation matrix as R ⁻¹ ,
With respect to the coordinates (bx, by, bz) of the point Pb, a point converted by the synthesis matrix is obtained as a point Pb ′, and the coordinates of the point Pb ′ are obtained as (bx ′, by ′, bz ′);
Based on the control signal from the controller, determine the angle θ,
Coefficient A '

Acquired by
The coordinates (x, y, z) of the viewpoint candidate point are

V '= (0,0,1)
Calculated by
The projection system according to claim 1. The first test image, when displayed on a display screen, includes the vertical reference line passing through a center point of the first test image,
The projection system according to claim 1. The second test image, when displayed on a display screen, includes the horizontal reference line passing through a center point of the second test image.
The projection system according to claim 1. The first test image is
In a state displayed on the display screen, is an image including a pattern that can determine the upper part of the image,
The projection system according to claim 1. When the viewpoint identification processing is performed,
The projection image adjustment unit,
Performing the viewpoint specifying process on a third test image including a grid pattern including a plurality of square grid patterns ;
The projection unit projects the third test image after the execution of the viewpoint identification processing on the projection plane ,
The projection system according to claim 1. The third test image is
A first grating pattern having a first pattern, a second grating pattern having a second pattern, in the absence geometrical distortion, the first direction on the third test image, and the second 3 is an image including a grid pattern formed by being alternately arranged in a second direction orthogonal to the first direction on the test image,
The projection system according to claim 6. A projector device used in the projection system according to any one of claims 1 to 7,
The projection unit, comprising the projection image adjustment unit,
Projector device. A projection method executed using a controller that generates a control signal based on a user operation, a plane parallel to a plane including a left-eye viewpoint and a right-eye viewpoint, and a right-eye cornea and a left-eye cornea. A projection method for projecting an image such that an inclined plane is a projection plane with respect to the plane that is a common tangent plane, and a geometric image distortion is reduced when viewed from a user's viewpoint. A program to be executed on a computer,
A projecting step of projecting an image on the projection plane, wherein a first test image having a vertical reference line defining a vertical direction of the image is projected, and a second test having a horizontal reference line defining a horizontal direction of the image is performed. Said projecting step of projecting an image;
A three-dimensional shape measuring step of measuring a three-dimensional shape of the projection plane;
When,
(1A) the three-dimensional shape measurement based on the three-dimensional shape data measured by the step, the Ru through the projection center point which is the intersection of the projection and the projection axis for projecting the image in step with the projection plane the projection when viewed from the first point is a point on the normal line of the surface, and such so that the state in which the geometric image distortion is reduced, which is the image conversion process with respect to the first test image correction process Run,
(2A) determining a first rotation angle according to the control signal, and projecting the first test image after the correction processing onto the projection surface when the projection surface is viewed from the first point by the projection step; Performing a first rotation process as an image conversion process on the first test image after the correction process so that the image is an image rotated by the first rotation angle around the projection center point;
(1B) the three-dimensional shape measurement based on the three-dimensional shape data measured by the step, the Ru through the projection center point which is the intersection of the projection and the projection axis for projecting the image in step with the projection plane the projection when viewed from the first point is a point on the normal line of the surface, and such so that the state in which the geometric image distortion is reduced, which is the image conversion process on the second test image correction processing Run,
(2B) determining a second rotation angle according to the control signal, and projecting the second test image after the correction processing onto the projection surface in the projection step when the projection surface is viewed from the first point. Performing a second rotation process as an image conversion process on the second test image after the correction process so that the image is an image rotated by the second rotation angle around the projection center point;
(3) a first straight line including the vertical reference line on the projection plane of the first test image after the first rotation process, and the projection of the second test image after the second rotation process; A viewpoint candidate point where the first straight line and the second straight line appear orthogonal to each other is determined based on a second straight line that is a straight line including the vertical reference line on a plane, and the determined viewpoint candidate point is determined. in so that such a state of geometrical image distortion is reduced when viewed from the perspective candidate points, to perform the viewpoint specific process of converting the image projected on the projection plane, a projection image adjustment step,
Equipped with a,
In the projection image adjusting step,
A line on the projection image of the first test image corresponding to the vertical reference line of the first test image is a vertical straight line when viewed from a user's viewpoint by executing the first rotation process. After the first state is reached, the second rotation process is executed,
Further, in the projection image adjusting step,
By executing the second rotation process, a line on the projection image of the second test image corresponding to the horizontal reference line of the second test image is horizontal when viewed from a user's viewpoint. After the second state, which is a straight line state, the viewpoint specifying process is executed,
Further, in the projection image adjusting step,
According to the control signal from the controller, the determined viewpoint candidate point set before the current time, a point included in the viewpoint candidate point, other than the determined viewpoint candidate point set before the current time By changing to the point of, the determined viewpoint candidate point is changed, and using the changed determined viewpoint candidate point, the viewpoint specifying process is executed,
A program for executing the projection method on a computer.

Images