JP2010128102A - Projection display device and positional relationship detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique to detect arrangement relationship between a projection display device which projects an image to a projection surface and the projection surface. <P>SOLUTION: A projector 100 includes an image compositing part 121 forming a first superposed image obtained by superposing an image projected by a projection optical system 150 on a first calibration image having predetermined shape, and a second superposed image obtained by superposing the image on a second calibration image different from the first calibration image in image property and having predetermined shape wherein at least a part of the shape is overlapped thereon. The projector 100 further includes a measurement image detection part 122 detecting a measurement image expressed by a difference of a pixel value of each pixel between a first picked-up image obtained by projecting the first superposed image to a screen SC and picking up the image by an imaging part 180 and a second picked-up image obtained by projecting the second superposed image to the screen SC and picking up the image by the imaging part 180. A measuring point is extracted from the measurement image, and the arrangement relationship between the projector 100 and the screen SC is detected based on the measuring point. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a method for detecting an arrangement relationship between a projection display device and a projection surface in a projection display device that projects an image onto a projection surface.

  Conventionally, in a projection display apparatus such as a projector using a light modulation device such as a liquid crystal panel for image formation, keystone distortion correction and focus adjustment of a projected image (hereinafter referred to as “projected image”) (hereinafter referred to as “projection image”) Technology for performing “adjustment for projection”) is known. When performing such adjustments for projection, a calibration image projected on a projection surface such as a screen is captured by an imaging unit such as a CCD camera provided in the projection display device, and the projection type is based on the captured image. The positional relationship between the display device and the projection surface is measured (for example, refer to the following patent document).

JP 2006-60447 A

  Therefore, when performing trapezoidal distortion correction, focus adjustment, etc., in order to facilitate the extraction of the calibration image from the image captured by the imaging unit, even when an image signal is input from the outside The projection of the image based on the image signal was interrupted, and only the calibration image was displayed.

  However, when performing such adjustment for projection, switching to the display of the calibration image usually involves a sudden change in the brightness of the screen, so it takes a long time until the imaging unit stably performs imaging. This causes a problem.

  The present invention has been made to solve the above-described conventional problems, and an object thereof is to provide a technique for performing an image based on an image signal without interruption when a measurement image is projected onto a projection surface. And

  In order to solve at least a part of the problems described above, the present invention can take the following forms or application examples.

[Application Example 1]
A projection display device,
A projection unit that projects an image onto the projection surface;
An imaging unit that images the projection surface on which the image is projected;
A first synthesized image output unit that synthesizes an image projected by the projection unit and a first calibration image having a predetermined shape and outputs the synthesized image to the projection unit as a first synthesized image;
A second composite image that is different from the first calibration image and has a predetermined shape that overlaps at least a part of the shape with the image, and outputs the composite image to the projection unit as a second composite image. An output section;
The first synthesized image is projected on the projection plane by the projection unit and captured by the imaging unit, and the second synthesized image is projected on the projection plane by the projection unit and captured by the imaging unit. A projection display device comprising: an arrangement relationship detection unit that extracts a measurement point from the second captured image and detects an arrangement relationship between the projection unit and the projection plane based on the measurement point.

  According to this projection display apparatus, a composite image obtained by combining a calibration image having a predetermined shape with the image to be projected is projected onto the projection surface and captured by the imaging unit. There is no significant change in the brightness of the image, and the imaging of the projection plane by the imaging unit is stably performed in a short time. The predetermined shape may be any shape that allows extraction of measurement points from the first and second captured images, such as a polygon such as a triangle or a square, a straight line, a cross shape, an annular shape, or a combination thereof. Various shapes can be employed. The predetermined shape is not limited to a single shape, and for example, a plurality of various shapes such as circles and squares may be arranged. The arrangement may be a matrix, a linear shape, an annular shape, an arrangement along a predetermined curve, or a random arrangement. Further, the plurality of shapes may be arranged apart from each other, or at least a part thereof may be a shape obtained by overlapping a part. Of course, one or a small number of complicated shapes may be used.

  Various points such as end points, corners, intersections, midpoints, center of gravity, and proration points can be used as measurement points by utilizing the properties of such shapes. When the predetermined shape is composed of an array of a plurality of shapes, the measurement points may be extracted from each shape, but the relationship between the plurality of shapes arranged, for example, an intermediate point between the shapes, virtually between the shapes You may extract from the intersection etc. of the drawn line segment.

[Application Example 2]
The projection display device according to Application Example 1 further includes a measurement image detection unit that detects a measurement image represented by a difference between pixel values of each pixel of the first captured image and the second captured image.
The arrangement relationship detecting unit extracts the measurement point based on the measurement image.

  According to this projection display apparatus, since the measurement image is detected based on the difference between the pixel values of the respective pixels of the first captured image and the second captured image, dust or the like adheres to the screen SC. Even if the dust appears in the captured image, the image corresponding to the dust is canceled when the measurement image is detected. Accordingly, it is possible to detect a measurement image with high accuracy.

[Application Example 3]
A projection display device according to Application Example 2,
The predetermined shape in which at least a part of the shape in the first and second calibration images overlap is as follows:
A projection display device in which the overlapping portion includes a shape that can extract three points that can define a plane.

  According to the projection display apparatus, the overlapping portion of the first calibration image and the second calibration image includes three points that can define a plane so that the first imaging including the shape of the first calibration image can be performed. The measurement image represented by the difference in pixel value of each pixel between the image and the second captured image including the shape of the second calibration image includes the shape of the overlapping portion of the first captured image and the second captured image. And three points that can define the plane, the plane can be defined from the measurement image.

[Application Example 4]
A projection display device according to Application Example 3,
The arrangement relationship detection unit is a projection display device that calculates a projection angle that is an inclination between an optical axis of projection light projected from the projection unit and the projection plane as one of the arrangement relationships.

  According to this projection display device, the arrangement relationship detecting unit calculates the projection angle based on the measurement image, and therefore, the user does not need to measure the projection angle and input the value to the projection display device. .

[Application Example 5]
The projection display device described in Application Example 4 further includes
A projection display device comprising a trapezoidal distortion correction unit that corrects a trapezoidal distortion of a projected image projected on the projection plane using the projection angle calculated by the arrangement relationship detection unit.

  According to the projection display device, since the projection display device includes the trapezoidal distortion correction unit, it is not necessary for the user to manually correct the trapezoidal distortion of the projection image by moving the projection display main body or the projection surface. .

[Application Example 6]
A projection display device according to Application Example 3,
The arrangement relation detection unit is a projection display apparatus that calculates a projection distance between the projection display apparatus and the projection plane as one of the arrangement relations.

  According to this projection display device, the arrangement relationship detection unit calculates the projection distance based on the measurement image, so that the user does not need to measure the projection distance and input the value to the projection display device. .

[Application Example 7]
The projection display device described in Application Example 6 further includes
A projection display device comprising a focus adjustment unit that performs focus adjustment of a projection image projected on the projection plane using the projection distance calculated by the arrangement relationship detection unit.

  According to this projection display apparatus, since this projection display apparatus includes the focus adjustment unit, it is not necessary for the user to manually adjust the focus of the projected image.

[Application Example 8]
A projection display device according to any one of Application Example 1 to Application Example 7,
The projection display apparatus, wherein the property of the image in the first and second calibration images is at least one of brightness, hue, and saturation.

  According to these projection display devices, the first calibration image and the second calibration image only need to differ in at least one of brightness, hue, and saturation. Many combinations are possible.

[Application Example 9]
A projection display device according to any one of Application Example 1 to Application Example 7,
The property of the image in the first and second calibration images is brightness,
The projection display apparatus, wherein the first calibration image is white and the second calibration image is black.

  According to these projection display devices, the first calibration image is white and the second calibration image is black, and the difference in brightness between the two images is significant. Therefore, the measurement that can be performed based on the two calibration images. The image for use becomes a high-contrast image, and errors in extracting measurement points from the image for measurement can be reduced. Further, the first calibration image may be black and the second calibration image may be white.

[Application Example 10]
An arrangement relation detection method for detecting an arrangement relation between a projection display device and a projection plane,
Combining a first calibration image having a predetermined shape with the image projected on the projection surface and outputting the first calibration image to the projection unit as a first composite image;
A step of synthesizing a second calibration image having a predetermined shape, which is different from the first calibration image and having at least a part of the shape, with the image and outputting the second calibration image to the projection unit as a second synthesized image;
A first captured image obtained by projecting the first composite image on the projection plane by the projection unit and captured by the imaging unit, and a second captured image projected by the projection unit on the projection plane by the projection unit. Detecting a measurement image represented by a difference in pixel value of each pixel from two captured images;
Extracting the measurement points from the detected image for measurement, and detecting the arrangement relationship between the projection display device and the projection plane based on the measurement points.

  According to this arrangement relation detection method, since the arrangement relation between the projection unit and the projection plane can be detected simply by projecting two composite images on the projection plane, the arrangement relation between the projection unit and the projection plane can be detected. Therefore, it is not necessary to project various measurement images on the projection surface.

  Note that the present invention can be realized in various modes. For example, the present invention can be realized in the form of a projection surface detection method and apparatus, a projection image correction method and apparatus, an integrated circuit for realizing the functions of the method or apparatus, a computer program, a recording medium on which the computer program is recorded, and the like. it can.

Next, embodiments of the present invention will be described based on examples.
A. First embodiment:
(A1) Overall configuration of projector (A2) Outline of correction of trapezoidal distortion (A3) Image detection process for measurement (A4) Three-dimensional surveying process (A5) Trapezoidal distortion correction process Second embodiment:
C. Variation:

A. First embodiment:
(A1) Overall configuration of the projector:
FIG. 1 is a block diagram showing an overall configuration of a projector 100 according to the first embodiment of the present invention. Projector 100 receives an image signal representing an image from the outside, and displays it as an image (hereinafter referred to as “projected image”) on a projection surface such as a screen SC. In this embodiment, the screen SC is almost upright, and the screen surface is rectangular.

  The projector 100 is roughly divided into an optical system for forming an optical image and an image processing system for electrically processing a video signal. The optical system includes an illumination optical system 140, a liquid crystal panel 130, and a projection optical system 150. The illumination optical system 140 includes a discharge lamp (not shown), and emits light from the discharge lamp as light used for projection. The liquid crystal panel 130 receives a signal from an image processing system, which will be described later, and forms an image on the panel surface. The liquid crystal panel 130 includes three liquid crystal panels corresponding to the three primary colors of RGB in order to perform color projection. Therefore, the light from the illumination optical system 140 is separated into three color lights of RGB, and each color light is incident on each corresponding liquid crystal panel. The color light modulated by passing through each liquid crystal panel is combined by a combining optical system such as a cross dichroic prism and emitted to the projection optical system 150.

  The projection optical system 150 includes a zoom lens 152 that performs enlargement / reduction of a projected image and a focus adjustment, a zoom adjustment motor 156 that adjusts the degree of zoom, and a focus adjustment motor 157 that adjusts the focus. Yes. The projection optical system 150 receives light modulated by the liquid crystal panel 130 and forms a projection image on the screen SC using the zoom lens 152. The zoom lens 152 is adjusted by a zoom adjustment motor 156 and a focus adjustment motor 157 to adjust the position of the lens, zoom adjustment for enlarging / reducing the projected image on the screen SC, and projecting the projected image on the screen SC. Focus adjustment is performed to properly form an image.

  On the other hand, the image processing system is composed mainly of the CPU 120 and the video processor 134 that are responsible for substantial overall processing. The A / D conversion unit 110, the liquid crystal panel driving unit 132, the zoom lens driving unit 155, the RAM 160, the calibration image. A ROM 170 including a storage unit 171, an imaging unit 180, a captured image memory 182, a remote controller control unit 190, a remote controller 191, and the like are provided. These elements constituting the image processing system are connected to each other via a bus 102.

  The A / D converter 110 is a device that performs A / D conversion on an input signal input from an image output device such as a personal computer or a DVD player via the cable 200, and converts the converted digital image signal into a video processor 134. Output to. The video processor 134 performs processing for adjusting the display state of the image, such as brightness, contrast, color density, hue, and projected image shape, on the input digital image signal, and then a liquid crystal panel driving unit. The processed video signal is output to 132. Based on this video signal, the liquid crystal panel drive unit 132 drives the liquid crystal panel 130. As a result, an image corresponding to the image signal input via the A / D converter 110 is formed on the liquid crystal panel 130, and this image is formed on the screen SC via the projection optical system 150. Become.

  Image processing performed by the video processor 134 includes trapezoidal distortion correction in addition to the above-described correction of brightness, contrast, hue, and the like. In FIG. 1, a circuit that performs trapezoidal distortion correction is shown as a trapezoidal distortion correction unit 136 in particular. The trapezoidal distortion correction unit 136 performs a trapezoidal distortion correction of the projected image on the digital image signal based on a projection angle value calculated by a projection angle calculation unit 126 described later in the CPU 120. The video processor 134 is also responsible for displaying a specific calibration image in the trapezoidal distortion correction. The video processor 134 may be a general-purpose processor sold as a DSP (digital signal processor) for correcting trapezoidal distortion, but may be configured as a dedicated ASIC.

  A CPU 120 that performs image processing in the projector 100 together with the video processor 134 includes an image composition unit 121, a measurement image detection unit 122, a zoom ratio calculation unit 123, a focal length calculation unit 124, and a three-dimensional survey unit 125. And a projection angle calculation unit 126. Each of these units is realized by the CPU 120 executing a specific program stored in the ROM 170 in advance. Each processing unit has a relative distance (hereinafter referred to as a projection distance) between the projector 100 and the screen SC, which will be described in detail later, and a projection angle (hereinafter referred to as an inclination of the screen SC with respect to the optical axis of the projection light projected from the projector). Processing necessary to calculate (projection projection angle) is performed.

  When the CPU 120 calculates the projection projection angle and the projection distance by the function of each unit, the CPU 120 outputs a signal corresponding to the projection projection angle to the video processor 134 and a signal corresponding to the projection distance to the zoom lens driving unit 155. When receiving a signal corresponding to the projection projection angle from the CPU 120, the video processor 134 performs trapezoidal distortion correction using the received signal. If the projection projection angle, which is the angle formed by the optical axis of the optical system of the projector 100 and the screen SC, is known, it can be calculated how the image is distorted, so the image processor 134 has a parameter corresponding to the projection projection angle. When the setting is made, the image input from the A / D converter 110 is compensated so as to correct the distortion of the projected image, and the compensated video signal is output to the liquid crystal panel driving unit 132.

  On the other hand, when the zoom lens driving unit 155 receives a signal corresponding to the projection distance from the CPU 120, the zoom lens driving unit 155 drives the focus adjustment motor 157 based on the signal to perform focus adjustment. In order to perform focus adjustment, it is necessary to know the zoom ratio of the zoom lens. In this embodiment, the zoom ratio is calculated from the drive amount of the zoom lens 152 by the zoom adjustment motor 156. Of course, it is also possible to calculate from an image captured by the imaging unit 180 described later.

  A work area necessary for the operation of the CPU 120 is secured on the RAM 160. The video processor 134 includes a work area necessary for executing each process, such as an image display state adjustment process performed by itself, as a built-in RAM. The ROM 170 stores two calibration images used for detection of a measurement image, which will be described later, in addition to the programs for realizing the above-described processing units, and corresponds to the calibration image storage unit.

  The remote control unit 190 receives an instruction from the user through the remote controller 191 and transmits it to the CPU 120 via the bus 102. In this embodiment, the projector 100 receives an instruction from the user through the remote control unit 190 and the remote controller 191, but may receive an instruction from the user through another configuration such as an operation panel provided in the projector 100. Good.

  The imaging unit 180 will be described. The imaging unit 180 is provided at a position where the front surface of the projector 100, that is, the direction in which the projection optical system 150 projects an image toward the screen SC can be imaged, and the image is projected onto the screen SC at a recommended projection distance. The camera direction and the angle of view are set so that the entire projected image is at least within the imaging range. The imaging unit 180 includes a CCD, which is a well-known image sensor, a single focus lens that forms an image on the CCD, a mechanism such as auto iris that adjusts the amount of light incident on the CCD, and a control circuit that reads a video signal from the CCD. . The auto iris mechanism receives a signal corresponding to the accumulated brightness value of the image from the CCD camera from the control circuit, and adjusts the iris (aperture) provided on the single focus lens so that the accumulated brightness value falls within a predetermined range. It adjusts automatically. The image whose brightness has been adjusted by auto iris is output from the imaging unit 180 to the captured image memory 182 and repeatedly written in a predetermined area of the captured image memory 182. When the captured image memory 182 completes the writing of the image for one screen, the flag of the predetermined area is sequentially inverted. Therefore, the CPU 120 refers to this flag to determine whether the imaging using the imaging unit 180 is completed. You can know whether or not. The CPU 120 accesses the captured image memory 182 while referring to this flag, and acquires a necessary captured image.

(A2) Outline of correction of trapezoidal distortion An outline of correction of trapezoidal distortion is shown in FIG. As shown in the figure, the trapezoidal distortion is corrected by a series of processes (steps S110 to S140) for detecting a measurement image for correcting the trapezoidal distortion, a three-dimensional measurement process (step S200), and actually. Calibration is performed from the process of correcting trapezoidal distortion (step S300). A series of processes in steps S110 to S140 corresponds to a process for detecting a measurement image. Details of these processes will be described later in detail with reference to FIGS. In the present embodiment, the process shown in FIG. 2 is started when the user operates a button on the remote controller 191 to instruct trapezoidal distortion correction. Of course, the measurement image detection process may be automatically executed, for example, when the power is turned on or the initial screen is projected thereafter.

  If a measurement image is obtained by a series of processes, a three-dimensional survey process (step S200) of the screen SC is performed based on the measurement image. This process will be described later with reference to FIGS. Next, a trapezoidal distortion correction process (step S300) is performed using the result of the three-dimensional survey of the screen SC. This process will be described later with reference to FIG.

(A3) Measurement image detection processing:
First, the measurement image detection process will be described. In order to correct the trapezoidal distortion, it is necessary to detect the positional relationship between the screen SC and the optical axis of the projector. Therefore, the measurement image includes a plurality of measurement points that can identify the surface of the screen SC. The CPU 120 finally uses these measurement points to calculate a projection distance that is a relative distance between the projector 100 and the screen SC, and a projection projection angle that is an inclination of the screen SC with respect to the optical axis of the projection light projected from the projector. Is calculated.

  When the process shown in FIG. 2 is started, the CPU 120 executes a first captured image acquisition process (step S110), and further executes a second captured image acquisition process (step S120). Details of these processes will be described with reference to FIGS. 3, 4, and 5. In the first captured image acquisition process (step S110), the CPU 120 temporarily freezes the projected image being projected by the projector 100 in the projected state (FIG. 4: step S112). In this process, the image input and projected via the A / D conversion unit 110 is converted into a still image at a specific timing, and the CPU 120 sends a predetermined command to the video processor 134. Is called. Still image (frozen image) projection continues until the video processor 134 receives a still image projection end command from the CPU 120. During this time, the CPU 120 reads the first calibration image 310 (see FIG. 3) stored in the calibration image storage unit 171 of the ROM 170, and outputs it to the video processor 134 together with the command to freeze it first. Processing for generating a first superimposed image in which the still image 300 (see FIG. 3) and the first calibration image 310 are superimposed is performed (step S114). FIG. 3 shows an example of the first superimposed image 315. As shown in the figure, in this example, the superimposed first calibration image 310 is nine white square images of 3 rows × 3 columns, and the first superimposed image 315 is nine white square regions. Is an image superimposed on the frozen still image.

  As a result of the generation of the first superimposed image by the CPU 120 (step S114), this image is projected onto the screen SC, and the CPU 120 performs processing for capturing the image projected onto the screen SC using the imaging unit 180 (step S114). Step S116). Specifically, the CPU 120 instructs the video processor 134 to project the first superimposed image, and the video processor 134 that has received this instruction sends the first superimposed image to the liquid crystal panel via the liquid crystal panel driving unit 132. To 130. Then, the imaging unit 180 is instructed to image at the timing when the first superimposed image is projected onto the screen SC. Upon receiving the imaging instruction, the imaging unit 180 adjusts the auto iris so that the brightness of the image incident on the CCD is suitable for imaging, and then performs imaging. The captured image (hereinafter referred to as the first captured image) is stored in the captured image memory 182 by the image capturing unit 180, but when the storage of the first captured image is completed, the flag is inverted. By monitoring the flag, it can be confirmed that the first captured image is stored in the captured image memory 182 (step S118). An example of the first captured image 317 stored in the captured image memory 182 is shown in FIG. If the optical axis of the projection optical system and the screen SC are not orthogonal to each other, as illustrated in FIG. 3, the first captured image 317 is an image in which trapezoidal distortion has occurred as compared to the first superimposed image 315.

  Next, the second captured image acquisition process will be described with reference to FIG. The CPU 120 acquires the second calibration image 320 (FIG. 3) from the calibration image storage unit 172 in the ROM 170 and causes the video processor 134 to superimpose the second calibration image 320 (FIG. 3) on the still image 300 frozen in the first calibration image acquisition process. Processing for instructing is performed (step S124). The difference from the first captured image acquisition process is that, in the second captured image acquisition process, the second calibration image 320 is superimposed instead of the first calibration image 310 (FIG. 3). As shown in FIG. 3, the second calibration image 320 is an image composed of black squares arranged in 3 rows × 3 columns at the same position as the first calibration image 310. Therefore, as shown in FIG. 3, the second superimposed image 325 obtained in step S124 is an image in which nine black square regions are superimposed on a still image.

  Thereafter, through the same processing steps as the first captured image acquisition process, the second superimposed image 325 is projected and captured (step S126), and the captured second captured image 327 (see FIG. 3) is captured image memory 182. (Step S128). Thereafter, a command for releasing the freeze of the projection image temporarily frozen in step S112 (FIG. 4) of the first captured image acquisition process is output to the video processor 134 (step S129), and the second captured image acquisition process is performed. Exit.

  By executing these two subroutines (first captured image acquisition process and second captured image acquisition process), the first captured image 317 and the second captured image 327 are within a predetermined address range of the captured image memory 182. , Will be in a saved state. Therefore, the CPU 120 next executes step S130 (FIG. 2) described above of the measurement image detection process, calculates the difference between the pixel values of the first captured image 317 and the second captured image 327, and calculates the difference value. A process of detecting the measurement image 330 (FIG. 3) represented by is performed (step S140). Since the measurement image 330 is an image obtained by calculating the difference between the pixel values of each pixel of the two captured images, all the shades and hues due to the frozen image are canceled and detected as an image including the measurement region 335. In this embodiment, an image including a black background portion and nine white portions of 3 rows × 3 columns as shown in FIG. 7A is detected as the measurement image 330. In the present embodiment, the first calibration image is white and the second calibration image is black. However, if at least one of brightness, hue, and saturation is different, the two calibration images are for measurement. An image obtained through the image detection process can be detected as a measurement image.

(A4) Three-dimensional survey processing:
Next, the three-dimensional survey process will be described. The three-dimensional survey process is a process for detecting a three-dimensional state of a plane including the screen SC in a three-dimensional coordinate system (hereinafter also referred to as “lens coordinate system”) having the principal point of the zoom lens 152 of the projector 100 as an origin. . That is, the three-dimensional inclination of the screen SC relative to the optical axis of the projection optical system in the projector 100 is detected.

  FIG. 6 is a flowchart showing the flow of the three-dimensional survey process in the projector 100. FIG. 7 shows an image used in the process of the three-dimensional surveying process. The three-dimensional survey process is automatically started after the measurement image detection process. When this process is started, the CPU 120 first performs a process of discretizing the previously detected measurement image 330 (step S205) and calculating nine measurement points 337 as a process of the three-dimensional surveying unit 125 (step S210). ). An example of a discretized image obtained by discretizing the measurement image 330 shown in FIG. 7A is shown in FIG. The discretization process used here refers to scanning the two-dimensional plane coordinates including the measurement image 330 in the horizontal direction and the vertical direction to obtain the gray level distribution of each pixel, and from the gray level distribution, three gravity center positions in each direction. Is obtained, and the center coordinates of the white square area are obtained (FIG. 7C). Even if the contours of the measurement image 330 and the measurement region 335 are blurred, the measurement point 337 in the measurement region 335 is uniquely determined by discretization. That is, the measurement image with a blurred outline is projected from the first captured image 317 and the second captured image 327 with a blurred outline projected and projected on the screen SC by the projector 100 in which the focus of the zoom lens 152 cannot be adjusted. Even if 330 is obtained, the measurement point 337 can be calculated. The coordinates of a total of nine measurement points P1 to P9 can be expressed as P1 (X11, Y11) to P9 (X33, Y33) (FIG. 7D).

  Next, a process of selecting three points that can define a plane among the nine measurement points 337 calculated in step S210 is performed (step S215). Regarding the selection of three points that can define a plane, any of the nine measurement points 337 may be selected as long as all three points are not on the same straight line. In this embodiment, nine measurement points 337 are detected because an obstacle or the like is interposed between the projection optical system 150 and the screen SC, and a part of the superimposed image projected from the projection optical system 150 is the screen SC. Even when an obstacle is interposed between the image capturing unit 180 and the screen SC and a part of the superimposed image cannot be captured, the measurement point 337 from a part of the captured measurement image 330 is displayed. This is because three-dimensional survey can be performed using the measurement point 337. In such a case, three-dimensional surveying is possible if at least one measurement point can be detected in each row and column among the nine measurement points 337 arranged in 3 rows × 3 columns in this embodiment.

  Next, the three-dimensional coordinates in the lens coordinate system of the three selected measurement points 337 are detected (step S220). The detection of the three-dimensional coordinates of the measurement point 337 is performed using an active active stereo method that detects the three-dimensional coordinates by using the parallax between the zoom lens 152 and the imaging unit 180 based on the principle of triangulation. Note that a method other than the active stereo method, such as a passive stereo method using a plurality of cameras, may be used to detect the three-dimensional coordinates of the measurement point.

Based on the three-dimensional coordinates of the three measurement points 337 thus detected, an approximate plane that approximates the plane including the screen SC is calculated (step S230). Specifically, an approximate plane equation that approximates a plane including the screen surface is calculated by the least square method using these three measurement points. After the calculation of the approximate plane equation, the three-dimensional survey process ends.

(A5) Keystone distortion correction processing:
Next, the trapezoidal distortion correction process will be described. The trapezoidal distortion is a trapezoidal distortion generated in the projected image from the projector 100 when the screen surface including the screen SC is not installed perpendicular to the direction of the optical axis of the projection light projected from the projector 100. It is. The trapezoidal distortion correction process corrects the distortion of the trapezoidal projection image.

  Details of the trapezoidal distortion correction process (FIG. 2: step S300) are shown in FIG. In the trapezoidal distortion correction process, the CPU 120 first executes a process as the projection angle calculation unit 126, thereby calculating the angle between the approximate plane of the screen plane detected by the three-dimensional survey process and the optical axis of the projection light projected from the projector 100. A certain projection angle is calculated (step S310). In this embodiment, the angle formed by the normal of the approximate plane and the optical axis of the projection light is calculated as the projection projection angle. Next, based on the calculated projection projection angle, processing as a trapezoidal distortion correction unit is executed to calculate target coordinates of the measurement point 337 where the projection projection angle is perpendicular to the screen plane (step S320). Finally, a process for calculating a conversion coefficient for converting the coordinates of the measurement point 337 into the target coordinates is performed (step S330), and a process for setting the coefficient in the video processor 134 is performed (step S340).

The video processor 134 converts the input digital image signal using the set coefficient, and outputs the converted result to the liquid crystal panel drive unit 132. The coefficient set in the video processor 134 is a coefficient used when three-dimensional vector calculation is performed on pixel position information (coordinates) of the input image signal.

  This three-dimensional vector calculation is executed to convert the coordinates of the pixel into coordinates of a position for correcting the trapezoidal distortion. When this coefficient is set, the video processor 134 repeats vector calculation for the coordinates of each pixel with respect to the input digital image signal. Thereby, the trapezoidal distortion correction process is realized.

  As described above, according to the projector 100 of the first embodiment, the three-dimensional survey of the screen SC is performed using the superimposed image obtained by superimposing the calibration image on the projected image being projected on the screen SC, and the trapezoid of the projected image is obtained. Since distortion correction is performed, there is no significant change in the brightness of the projection surface between when the projected image is projected and when the superimposed image is projected, and the imaging of the projection surface by the imaging unit can be performed in a short time. This is because the image is not switched to a measurement-dedicated image, and it does not take time for the auto iris or the like to adjust the amount of received light. Further, since it is not affected by the state of the projected image being projected, it is possible to correct the trapezoidal distortion of the projected image without interrupting the viewed image or the like even while the viewed image or the like is being projected. Furthermore, since the measurement image 330 used for the three-dimensional survey is detected by the difference in pixel value of each pixel of the two captured images, the appreciation image or the like included in the captured image is canceled out, and the measurement region 335 is measured. It can be detected as an image that leaves only In other words, even if dust or the like adheres to the screen SC and the dust appears in the captured image, the image corresponding to the dust is canceled when the measurement image is detected. Image detection and subsequent trapezoidal distortion correction are possible.

  In the present embodiment, the image composition unit 121 corresponds to the first composite image output unit and the second composite image output unit in the claims, and the three-dimensional survey unit 125, the focal length calculation unit 124, and the projection angle calculation unit 126 include The liquid crystal panel 130, the illumination optical system 140, and the projection optical system 150 correspond to the projection unit in the claims, and other than that, the same as the claims in the present embodiment. The name corresponds to that of the claims.

B. Second embodiment:
Next, a second embodiment of the present invention will be described. In the first embodiment, as the positional relationship between the projection unit and the projection surface, a projection angle that is an angle between the approximate plane of the screen plane and the optical axis of the projection light projected from the projector 100 is calculated, and based on the projection angle. In the second embodiment, the projection distance, which is the distance between the approximate plane of the screen plane and the projection unit, is calculated as the positional relationship between the projection unit and the projection plane. The focus of the projected image is adjusted based on the projection distance. Further, the configuration of the projector is the same as that of the projector 100 of the first embodiment.

  Since the configuration of the projector in this embodiment is the same as that of the projector 100 in the first embodiment, this embodiment will be described with reference to FIG. 1 for the configuration and FIG. 9 for the flow of focus adjustment. The focus adjustment is performed by the CPU 120 in the projector 100 in FIG. 1, the zoom lens driving unit 155 that operates according to a command from the CPU 120, and the focus adjustment motor 157.

  FIG. 9 is a flowchart showing the flow of focus adjustment processing executed in the CPU 120. The focus adjustment is performed based on the approximate plane of the screen plane calculated in the first embodiment. Therefore, each process (steps S110 to S200) in FIG. 2 until the approximate plane is calculated is the same as the trapezoidal distortion correction, and is omitted from FIG.

  When the CPU 120 completes the processing from step S110 to step S200, the CPU 120 performs processing for calculating a projection distance that is a distance from the principal point of the zoom lens 152 to the screen SC based on the next approximate plane of the screen plane calculated. This is performed (step S410). This corresponds to the processing of the focal length calculation unit 124 shown in FIG. Thereafter, the focal length calculation unit 124 calculates a focal length at which the projection light from the projector 100 forms an image as a projection image on the screen SC based on the projection distance (step S420). Finally, an appropriate position of the zoom lens 152 and a drive amount of the focus adjustment motor 157 for moving the lens to the position are calculated so that the calculated focal length is in focus, and the value of the drive amount is calculated as the zoom lens driving unit 155. (Step S430), and the focus adjustment process is terminated. The zoom lens drive unit in which the drive amount value is set drives the focus adjustment motor 157 based on the drive amount to move the zoom lens 152 to an appropriate position, and the projection light is converted into a projection image on the screen SC. Focus and adjust the focus. Although it is necessary to read the zoom ratio of the zoom lens at the time of executing the focus adjustment processing, the zoom ratio is calculated from the driving amount of the zoom lens 152 by the zoom adjustment motor 156. Of course, in the present embodiment, as described above, it is also possible to calculate from an image captured by the imaging unit 180.

As described above, according to the projector of the second embodiment, when performing the focus adjustment, the content of the projected image is not limited, and the projected image can be performed without interruption. In addition, the superimposed image projected on the projection plane, which is performed when the measurement image is detected, is stably captured in a short time. Furthermore, since the measurement point can be accurately detected based on the measurement image, highly accurate focus adjustment can be performed.
The focus adjustment described as the second embodiment may be performed alone or together with the trapezoidal distortion correction described in the first embodiment.

C. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be carried out in various modes without departing from the gist thereof. For example, the following modifications are possible.

(C1) Modification 1:
In the above-described embodiment, the measurement image is detected in the projected image such as the projected appreciation image. However, the projected image is not limited to a still image but may be a moving image. The measurement image can be detected by detecting the measurement image by temporarily stopping the moving image in response to a projection image correction instruction from the user via the remote controller 191.

(C2) Modification 2:
In the above-described embodiment, the first calibration image 310 and the second calibration image 320 made of a figure having a specific shape arranged in 3 rows × 3 columns shown in FIG. 3 are used. Calibration images of other shapes may be used. An example of another calibration image is shown in FIG. As shown in the figure, a white grid-like third superimposed image 355 and a black grid-like fourth superimposed image 365 are created, and a third captured image 357 and a fourth captured image 367 are obtained by projection and imaging. Then, the measurement image 370 may be detected by obtaining the difference value of each pixel. In the first embodiment, the measurement point 337 is discretized by the gradation of the pixel value with respect to the two-dimensional plane of the measurement image 330 and the measurement point 337 is detected. However, in this modification, the measurement point is detected in the measurement region as the measurement point detected from the measurement image 370. What is necessary is just to let the intersection of the vertical line and horizontal line of a certain grid-like white line be a measurement point. In that case, the outline of the white line can be detected by applying an outline extraction filter such as a differential filter or a Laplacian filter to the white shade as white line outline detection means. In addition, as a calibration image, the shape is not limited to a square or grid, but may be a circle or a triangle, or a calibration image in which these shapes are arranged in N rows × M rows (N, M: natural numbers). But it ’s okay. Furthermore, the first calibration image and the second calibration image may not have the same shape. It suffices if the first calibration area and the second calibration area partially coincide with each other, and it is only necessary that the measurement area in the measurement image can be detected from the coincident shape. The shape of the calibration image is not limited to a geometric shape, and the coordinates of a specific point may be determined. For example, the image may include a significant image like an animation or a figure that can be read as a character. The predetermined shape described in the claims may indicate a single figure such as a square, a circle, or a triangle, or may include a single predetermined shape including all the figures in the calibration image. In the first and second embodiments, a single predetermined shape including all the figures in the calibration image is used, and in a modified example, a lattice-like figure alone is set to a predetermined shape. Even in an image including a significant image like an animation or a figure with a shape that can be read as a character, when those images include a plurality of figures, a single figure included in the image may be a predetermined shape, Of course, all the figures included in the image may be one predetermined shape.

In addition, although the projector 100 uses the liquid crystal panel 130 as a light modulation element, it may be a DLP projector using a DMD (digital mirror device) as the light modulation element. Further, the image sensor used in the imaging unit 180 may be a CMOS instead of the CCD. Furthermore, the light source used in the illumination optical system 140 may be an LED instead of a discharge lamp.
In the above embodiment, the angle formed by the normal of the approximate plane and the optical axis of the projection light is calculated as the projection projection angle. However, if the projection projection angle is an angle at which trapezoidal correction processing can be appropriately performed, good.

  In addition, in the above-described embodiments, a part of functions realized by software may be realized by hardware, or a part of functions realized by hardware may be realized by software.

It is a block diagram which shows the structure of the projector 100 in 1st Example. It is a flowchart which shows the trapezoid distortion correction in 1st Example. It is explanatory drawing which showed the image used for trapezoid distortion correction. It is a flowchart which shows the 1st captured image acquisition process in 1st Example. It is a flowchart which shows the 2nd captured image acquisition process in 1st Example. It is a flowchart which shows the three-dimensional survey process in 1st Example. It is explanatory drawing which showed the image used for a three-dimensional survey process. It is a flowchart which shows the trapezoid distortion correction process in 1st Example. It is a flowchart which shows the focus adjustment process in 2nd Example. It is explanatory drawing which showed the image used for the modification 2.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Projector 102 ... Bus 110 ... A / D conversion part 120 ... CPU
DESCRIPTION OF SYMBOLS 121 ... Image composition part 122 ... Measurement image detection part 123 ... Zoom ratio calculation part 124 ... Focal length calculation part 125 ... Three-dimensional surveying part 126 ... Projection angle calculation part 130 ... Liquid crystal panel 132 ... Liquid crystal panel drive part 134 ... For image Processor 136 ... Trapezoidal distortion correction unit 140 ... Illumination optical system 150 ... Projection optical system 152 ... Zoom lens 155 ... Zoom lens drive unit 156 ... Zoom adjustment motor 157 ... Focus adjustment motor 160 ... RAM
170 ... ROM
171: Calibration image storage unit 180 ... Imaging unit 182 ... Captured image memory 190 ... Remote control unit 191 ... Remote control 200 ... Cable 300 ... Still image 310 ... First calibration image 315 ... First superimposed image 317 ... First captured image 320 ... Second calibration image 325 ... second superimposed image 327 ... second captured image 330,370 ... measurement image 335 ... measurement region 337 ... measurement point 340 ... discrete image 355 ... third superimposed image 357 ... third captured image 365 ... Fourth superimposed image 367 ... Fourth captured image SC ... Screen

Claims (10)

A projection display device,
A projection unit that projects an image onto the projection surface;
An imaging unit that images the projection surface on which the image is projected;
A first synthesized image output unit that synthesizes an image projected by the projection unit and a first calibration image having a predetermined shape and outputs the synthesized image to the projection unit as a first synthesized image;
A second composite image that is different from the first calibration image and has a predetermined shape that overlaps at least a part of the shape with the image, and outputs the composite image to the projection unit as a second composite image. An output section;
The first synthesized image is projected on the projection plane by the projection unit and captured by the imaging unit, and the second synthesized image is projected on the projection plane by the projection unit and captured by the imaging unit. A projection display device comprising: an arrangement relationship detection unit that extracts a measurement point from the second captured image and detects an arrangement relationship between the projection unit and the projection plane based on the measurement point. The projection display device according to claim 1 further includes a measurement image detection unit that detects a measurement image represented by a difference between pixel values of each pixel of the first captured image and the second captured image,
The arrangement relationship detecting unit extracts the measurement point based on the measurement image. The projection display device according to claim 2,
The predetermined shape in which at least a part of the shape in the first and second calibration images overlap is as follows:
A projection display device in which the overlapping portion includes a shape that can extract three points that can define a plane. The projection display device according to claim 3,
The arrangement relationship detection unit is a projection display device that calculates a projection angle that is an inclination between an optical axis of projection light projected from the projection unit and the projection plane as one of the arrangement relationships. The projection display device according to claim 4 further includes:
A projection display device comprising a trapezoidal distortion correction unit that corrects a trapezoidal distortion of a projected image projected on the projection plane using the projection angle calculated by the arrangement relationship detection unit. The projection display device according to claim 3,
The arrangement relation detection unit is a projection display apparatus that calculates a projection distance between the projection display apparatus and the projection plane as one of the arrangement relations. The projection display device according to claim 6 further includes:
A projection display device comprising a focus adjustment unit that performs focus adjustment of a projection image projected on the projection plane using the projection distance calculated by the arrangement relationship detection unit. A projection display device according to any one of claims 1 to 7,
The projection display apparatus, wherein the property of the image in the first and second calibration images is at least one of brightness, hue, and saturation. A projection display device according to any one of claims 1 to 7,
The property of the image in the first and second calibration images is brightness,
The projection display apparatus, wherein the first calibration image is white and the second calibration image is black. An arrangement relation detection method for detecting an arrangement relation between a projection display device and a projection plane,
Combining a first calibration image having a predetermined shape with the image projected on the projection surface and outputting the first calibration image to the projection unit as a first composite image;
A step of synthesizing a second calibration image having a predetermined shape, which is different from the first calibration image and having at least a part of the shape, with the image and outputting the second calibration image to the projection unit as a second synthesized image;
A first captured image obtained by projecting the first composite image on the projection plane by the projection unit and captured by the imaging unit, and a second captured image projected by the projection unit on the projection plane by the projection unit. Detecting a measurement image represented by a difference in pixel value of each pixel from two captured images;
Extracting the measurement points from the detected image for measurement, and detecting the arrangement relationship between the projection display device and the projection plane based on the measurement points.

Images