JP2011199717A - Projection type display device and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology relating to trapezoidal distortion correction in a projection type display device for projecting an image toward a projection surface.SOLUTION: A projection type display device for projecting and displaying an image toward a projection surface is provided and includes: a projection means for projecting an image; an imaging means for imaging the projection surface to create a captured image of the projection surface; an adjustment means for adjusting the projection means into any state different from a focused state; a control means for controlling the adjustment means so as to adjust the projection means into any state different from the focused state before the imaging means images the projection surface; and a correction means for detecting a shape of the projection surface based on the captured image of the projection surface obtained by imaging with the projection means in any state different from the focused state, and correcting the image so as to be matched to the shape of the projection surface. Consequently erroneous detection of a contour of a projection image as framing of a screen can be suppressed.

Description

  The present invention relates to a projection display device that projects an image onto a projection surface, and an image display method in the projection display device.

  When an image is displayed on a screen using a projection display device such as a projector, the image projected on the screen (hereinafter referred to as “projection image”) is distorted due to the relative angle between the projection display device and the screen. May occur. In such a case, a technique for correcting an input image so that a projected image has an appropriate shape on a screen (hereinafter referred to as “trapezoidal distortion correction”) is known.

  When performing such trapezoidal distortion correction, the projection display device captures a screen with an imaging unit such as a built-in camera and detects a frame border of the captured image as a boundary line. Then, the projection display apparatus corrects the input image using the detected frame edge of the screen so that the shape of the projected image matches the shape of the screen. At this time, the projection display apparatus detects a frame edge of the screen in the photographed image by performing contour extraction processing on the photographed image to extract a contour included in the image based on a change in luminance or the like (see the following patent document). ).

JP-A-2005-318652 JP 2006-060447 A

  However, in the above prior art, among images included in the captured image, an image that is not actually an image of the frame of the screen may be erroneously detected as an image of the frame of the screen. In some cases, there has been a problem that appropriate trapezoidal distortion correction is not performed. For example, as described below, the projection display apparatus may erroneously detect the outline of the projected image and the frame edge of the screen.

If the projection display device is projecting a white image (hereinafter referred to as a “white image”) when the imaging unit captures the screen, the projection display device mistakenly outlines the white image as the frame of the screen. It may be detected. For this reason, when the screen is imaged by the imaging unit in order to detect the frame edge of the screen, the projection display device projects a black image (hereinafter referred to as “black image”).
When projecting this black image, the projection display device shields light from the light source using a light modulation element such as a liquid crystal panel (in other words, prevents light from the light source from entering the projection lens). ) To express a black image on the screen. However, due to the characteristics of the light modulation element itself and optical problems such as stray light, a phenomenon that the black image actually reproduced on the screen has a slight brightness (hereinafter referred to as “black floating”) may occur. . When this black float occurs, even if the projection image is a black image, the area occupied by the projection image on the screen is slightly brighter than the surrounding area. For this reason, even when the screen is captured with a black image projected from the projection display device, the projection display device erroneously detects the outline of the projection image as the frame edge of the screen and corrects the keystone distortion. May not be performed properly.
In particular, when the projection distance from the projection display device to the screen is short, when the environment for viewing the projected image is dark, or when the projection display device has high brightness, the projection display device displays the contour of the projection image on the screen. Easily detected as a frame edge.

  The present invention has been made to solve the above-described conventional problems. When the projection display apparatus detects a frame border of a screen from a captured image obtained by photographing the screen, the outline of the projection image is displayed on the screen. It aims at providing the technique which suppresses detecting erroneously with a frame side.

  In order to solve at least a part of the problems described above, the present invention can take the following forms or application examples.

  Application Example 1 A projection display device according to this application example is a projection display device that projects and displays an image on a projection surface, and projects projection means for projecting the image, and images the projection surface. An imaging unit that generates a captured image of the projection plane, an adjustment unit that adjusts the projection unit to a state different from the in-focus state, and the projection unit before the imaging unit captures the projection plane. Control means for controlling the adjusting means to adjust to a state different from the in-focus state, and the captured image of the projection plane imaged when the projection means is in a state different from the in-focus state. And a correction unit that detects the shape of the projection surface and corrects the image so as to match the shape of the projection surface.

  According to this application example, before the imaging surface is imaged by the imaging unit, the adjusting unit is adjusted to a state different from the in-focus state by the adjusting unit. When the projection unit is adjusted to a state different from the in-focus state, the projection unit is out of focus, so that the projected image projected from the projection unit has a blurred outline on the projection plane. When the projection plane is imaged with the outline of the projected image blurred, the outline of the projected image included in the captured image is more blurred than the outline of the projected image when the projection means is in focus. In the processing, it is less likely that the outline of the projected image is erroneously detected as the frame edge of the screen. Therefore, it is possible to suppress erroneous detection of the outline of the projected image as the frame edge of the screen.

  Application Example 2 In the projection display device according to the application example, the projection unit projects an adjustment image, and the imaging unit images the projected adjustment image to generate a captured image of the adjustment image. The control unit determines the in-focus state of the projection unit based on the captured image of the adjustment image, and the projection unit is in focus before the imaging unit images the projection plane. It is preferable to control the adjusting means so as to adjust to a state different from the state.

  According to this application example, the state in which the projection unit is in focus can be grasped. Therefore, when the projection unit is adjusted by the adjustment unit, the projection unit can be adjusted on the basis of the focus state. It becomes easy to adjust the means to a state different from the in-focus state.

  Application Example 3 In the projection display device according to the application example, the control unit controls the adjustment unit to adjust the projection unit to the in-focus state after the imaging unit images the projection surface. It is preferable to control.

  According to this application example, the control unit controls the adjustment unit so that the projection unit adjusts the projection unit to the in-focus state after the imaging unit captures the projection plane, and thus the projection unit projects an image after imaging the projection plane. When projecting onto a surface, the image can be displayed in a focused state.

  Application Example 4 In the projection display device according to the application example, the projection unit projects a measurement image, and the imaging unit generates a captured image of the measurement image by capturing the projected measurement image. The correction unit preferably corrects the input image based on the captured image of the projection plane and the captured image of the measurement image to generate a corrected image.

  According to this application example, the projection display device captures the projected measurement image and generates a captured image of the measurement image. If the measurement image is an image including a predetermined pattern, the projection display apparatus can determine the correspondence between coordinates defined on the input image and coordinates defined on the captured image. The projection display apparatus can correct the input image by using this correspondence.

  Application Example 5 In the projection display device according to the application example, the correction unit calculates a relative angle of the projection surface with respect to the projection display device based on a captured image of the projection surface, and the relative angle is calculated. It is preferable to correct the input image based on the above and generate a corrected image.

  According to this application example, the projection display device calculates the relative angle of the projection surface with respect to the projection display device based on the captured image of the projection surface, and corrects the input image based on the relative angle. Therefore, the projection display apparatus can correct the input image without using the measurement image.

  Application Example 6 An image display method according to this application example is an image display method of a projection display device that projects and displays an image on a projection surface, and focuses the projection unit of the projection display device. When the projection means is in a state different from the in-focus state, the projection surface is imaged to generate a captured image, and the shape of the projection surface is detected based on the captured image. The image is corrected so as to match the shape of the projection surface.

  According to this application example, the projection unit of the projection display apparatus is adjusted to a state different from the in-focus state before imaging the projection plane. When the projection unit is adjusted to a state different from the in-focus state, the projection unit is out of focus, so that the projected image projected from the projection unit has a blurred outline on the projection plane. When the projection plane is imaged with the outline of the projected image blurred, the outline of the projected image included in the captured image is more blurred than the outline of the projected image when the projection means is in focus. In the processing, it is less likely that the outline of the projected image is erroneously detected as the frame edge of the screen. Therefore, it is possible to suppress erroneous detection of the outline of the projected image as the frame edge of the screen.

  Note that the present invention can be realized in various modes. For example, the present invention can be realized in the form of a projection surface detection method and apparatus, a projection image correction method and apparatus, an integrated circuit for realizing the functions of the method or apparatus, a computer program, a recording medium on which the computer program is recorded, and the like. it can.

It is a block diagram which shows the structure of the projector 100 in an Example. It is a flowchart which shows the trapezoid distortion correction process in an Example. It is a flowchart which shows the relative position adjustment process in an Example. It is explanatory drawing which showed the focus adjustment image 310 used by the relative position adjustment process, and the guide display image 320. FIG. It is a flowchart which shows the captured image generation process in an Example. It is explanatory drawing which showed the black float which arose in the projected black image in the state which is not performing the focus blurring process in an Example. It is explanatory drawing which showed the result of having performed the boundary line detection process in the case where the focus blurring process in the Example is not performed. It is explanatory drawing which showed the black float which arose in the projected black image in the state which performed the focus blurring process in an Example. It is explanatory drawing which showed the result of having performed the boundary line detection process in the case of performing the focus blurring process in an Example. It is a flowchart which shows the correction | amendment shape determination process in an Example. It is explanatory drawing which showed coordinates Pc1-Pc4 of the four corners of the screen SC in a camera coordinate. It is explanatory drawing which showed the measurement image 330 used by correction | amendment shape determination processing. It is explanatory drawing which showed typically that a panel coordinate and a camera coordinate are matched by the projection transformation formula M1. It is explanatory drawing which showed the process which converts coordinate Pc1-Pc4 into coordinate Pp1-Pp4 by projective transformation formula M1. It is explanatory drawing which showed typically that the four corner coordinates of the input image in panel coordinates and coordinates Pp1-Pp4 are matched by the projective transformation formula M2.

Next, embodiments of the present invention will be described based on examples.
A. Example:
(A1) Configuration of projector (A2) Trapezoidal distortion correction processing (A3) Captured image generation processing (A4) Correction shape determination processing Modified example

A. Example:
(A1) Configuration of Projector FIG. 1 is a block diagram showing an overall configuration of a projector 100 according to an embodiment of the present invention. The projector 100 is a projection display device that receives an image signal representing an image from the outside and displays it as a projection image on a projection surface such as a screen SC. In the present embodiment, the projection surface shape of the screen SC is a rectangular shape.

  The projector 100 is roughly divided into an optical system that forms an optical image and an image processing system that electrically processes an image signal. The optical system includes an illumination optical system 140, a light modulation element 130, and a projection optical system 150, and projects an image onto the screen SC. This optical system corresponds to the projection means in the claims.

  The illumination optical system 140 includes a light source (not shown) that emits light. The illumination optical system 140 also includes an integrator optical system that uniformizes the illuminance distribution of the light emitted from the light source, and a polarization conversion optical system that converts the light emitted from the light source into polarized light having a predetermined polarization direction. May be.

  The light modulation element 130 forms an image based on a signal from an image processing system described later. In this embodiment, a description will be given assuming that a transmissive liquid crystal light valve is used as the light modulation element 130. The light modulation element 130 includes three liquid crystal light valves corresponding to the three primary colors of RGB in order to perform color projection. Therefore, the light from the illumination optical system 140 is separated into three color lights of R (red), G (green), and B (blue), and each color light enters each corresponding liquid crystal light valve. The color lights modulated by the liquid crystal light valves are combined by a combining optical system such as a cross dichroic prism and emitted to the projection optical system 150.

  The projection optical system 150 includes a zoom lens 152. The zoom lens 152 includes a plurality of lenses, and forms an image formed by the light modulation element 130 as a projection image on the screen SC. The zoom ratio when the zoom lens 152 projects a projected image and the focal length of the zoom lens 152 can be adjusted by a zoom lens driving unit 155 described later.

  On the other hand, the image processing system is configured around a CPU 120 and a video processor 134 that are responsible for substantial overall processing, and includes an A / D conversion unit 110, a light modulation element driving unit 132, a zoom lens driving unit 155, a RAM 160, a ROM 170, An imaging unit 180, a captured image memory 182 and a remote control unit 190 are provided. Each element constituting the image processing system is connected to each other via a bus 102.

  The A / D conversion unit 110 is a device that performs A / D conversion on an input signal input from an image output device such as a personal computer or a DVD player via the cable 200, and converts the converted digital image signal into a video processor. It outputs to 134. The video processor 134 performs processing for adjusting the luminance, contrast, color density, hue, shape of the projected image, and the like on the input digital image signal, and then performs the processing on the light modulation element driving unit 132. The processed image signal is output. Based on the image signal, the light modulation element driving unit 132 drives the light modulation element 130. As a result, an image corresponding to the image signal input via the A / D conversion unit 110 is formed by the light modulation element 130, and this image is formed on the screen SC via the projection optical system 150. become.

  Image processing performed by the video processor 134 includes trapezoidal distortion correction in addition to the above-described correction of brightness, contrast, hue, and the like. In FIG. 1, a circuit that performs trapezoidal distortion correction is shown as a trapezoidal distortion correction unit 136 in particular. The trapezoidal distortion correction unit 136 performs trapezoidal distortion correction of the projected image on the digital image signal. More specifically, the trapezoidal distortion correcting unit 136 performs a trapezoidal distortion correction on the input image based on the frame edge of the screen detected by the boundary detection unit 123 described later, and a corrected image whose shape is corrected. Is output. The video processor 134 may be a general-purpose processor sold as a DSP (digital signal processor) for correcting trapezoidal distortion, but may be configured as a dedicated ASIC. The trapezoidal distortion correction unit 136 corresponds to correction means in the claims.

  The CPU 120 includes a boundary line detection unit 123, a zoom lens adjustment unit 124, and a determination unit 125, and performs image processing in the projector 100. The boundary detection unit 123, the zoom lens adjustment unit 124, and the determination unit 125 are realized by the CPU 120 executing a specific program stored in advance in the ROM 170. The CPU 120 corresponds to the control means in the claims.

  The boundary line detection unit 123 detects the frame side of the screen SC as a boundary line from the captured image of the screen SC generated by the imaging unit 180 described later. The zoom lens adjustment unit 124 controls the zoom lens driving unit 155. More specifically, the zoom lens adjustment unit 124 outputs a zoom lens drive signal for driving the zoom lens 152 to a zoom lens drive unit 155 described later, and performs zoom adjustment and focus adjustment of the zoom lens 152. Do. The determination unit 125 determines whether or not the zoom lens 152 is in focus (in-focus state). More specifically, the determination unit 125 determines that a state where the contrast of the captured image obtained by capturing the projected focus adjustment image 310 (see FIG. 4) is maximum is the in-focus state.

  The zoom lens driving unit 155 includes a zoom adjustment motor 156 and a focus adjustment motor 157. The zoom lens drive unit 155 performs zoom adjustment for enlarging or reducing the projected image by adjusting the position of the lens constituting the zoom lens 152 by the zoom adjustment motor 156. The zoom lens driving unit 155 performs focus adjustment for adjusting the in-focus position of the zoom lens 152 by adjusting the position of the lens constituting the zoom lens 152 by the focus adjustment motor 157. The zoom lens driving unit 155 corresponds to the adjusting unit in the claims.

  When the zoom lens driving unit 155 receives a zoom lens driving signal from the CPU 120, the zoom lens driving unit 155 drives the zoom adjustment motor 156 and the focus adjustment motor 157 on the basis of the signal to perform zoom adjustment and focus adjustment.

  A work area necessary for the operation of the CPU 120 is secured on the RAM 160. Further, the ROM 170 stores a focus adjustment image 310, a guide display image 320, and a measurement image 330 that are used for trapezoidal distortion correction, which will be described later, in addition to the programs that implement the processing units described above.

  The remote control control unit 190 receives an instruction from the user through the remote control 191 and transmits it to the CPU 120 via the bus 102. The remote controller 191 includes an adjustment start button 192 and an adjustment end button 193. The user operates an adjustment start button 192 and an adjustment end button 193 when instructing the start and end of trapezoidal distortion correction described later. In this embodiment, the projector 100 receives an instruction from the user via the remote controller 191 by the remote control unit 190. However, the projector 100 may receive an instruction from the user through another configuration such as an operation panel provided in the projector 100. Good.

  The imaging unit 180 is provided in front of the projector 100, that is, at a position where the projection optical system 150 can capture an image in the direction of projecting an image toward the screen SC, and is projected onto the screen SC at a recommended projection distance. The camera direction and angle of view are set so that the entire projected image falls within the imaging range. The imaging unit 180 includes a known CCD, a single focus lens that forms an image on the CCD, a mechanism such as an auto iris that adjusts the amount of light incident on the CCD, and a control circuit that reads an image signal from the CCD. The auto iris mechanism receives a signal corresponding to the cumulative value of the image brightness from the CCD camera from the control circuit, and adjusts the iris (aperture) provided in the single focus lens so that the cumulative value of brightness falls within a predetermined range. It adjusts automatically. The image whose brightness has been adjusted by auto iris is output from the imaging unit 180 to the captured image memory 182 and repeatedly written in a predetermined area of the captured image memory 182. When the captured image memory 182 completes the writing of the image for one screen, the flag of the predetermined area is sequentially inverted. Therefore, the CPU 120 refers to this flag to determine whether the imaging using the imaging unit 180 is completed. It can be determined whether or not. The CPU 120 accesses the captured image memory 182 while referring to this flag, and acquires a necessary captured image. The imaging unit 180 corresponds to the imaging unit in the claims.

(A2) Trapezoidal Distortion Correction Process FIG. 2 shows the flow of the trapezoidal distortion correction process performed by the projector 100. The trapezoidal distortion correction process is a process of correcting the trapezoidal distortion of the projected image so that each side of the outer peripheral line of the projected image on the screen SC is parallel to each side of the frame of the screen SC. The trapezoidal distortion correction process is executed in response to an instruction from the user through the remote controller 191. The trapezoidal distortion correction process may be automatically executed according to, for example, power-on or an image signal input.

  In the trapezoidal distortion correction process, a relative position adjustment process (step S100), a screen imaging process for generating a captured image of the screen SC (step S200), and a frame edge of the screen SC in the captured image of the screen SC are detected as a boundary line. Boundary line detection processing (step S300), correction shape determination processing (step S400) for determining a correction shape based on the detected frame side of the screen SC, and correction for correcting the image signal based on the determined correction shape Processing (step S500).

  The relative position adjustment process (step S100) is a process that prompts the user to adjust the relative position of the projector 100 with respect to the screen SC. FIG. 3 shows the flow of processing performed in the relative position adjustment processing (step S100).

  In the relative position adjustment process (step S100), prior to projection, the zoom lens adjustment unit 124 outputs a zoom lens drive signal to the zoom lens drive unit 155 to adjust the focus of the projection optical system 150 including the zoom lens 152 ( Step S110). In this adjustment, the zoom lens adjustment unit 124 changes the focus of the zoom lens 152 in a state where, for example, the focus adjustment image 310 shown in FIG. At the same time, the zoom lens adjustment unit 124 causes the imaging unit 180 to image the projected focus adjustment image 310. Then, the determination unit 125 determines that the state in which the contrast of the projected focus adjustment image 310 is maximum is a state in which the zoom lens 152 is in focus (in-focus state). The determination unit 125 stores the state of the zoom lens 152 in the focused state in the RAM 160. The focus adjustment image 310 is stored in the ROM 170. The focus adjustment image 310 is desirably an image having a large luminance difference between a dark region and a bright region, and an image including a white background region and a black pattern is preferably used as shown in FIG. The focus adjustment image 310 corresponds to the adjustment image in the claims.

  When the focus adjustment is completed, the CPU 120 causes the projector 100 to project a guide display image 320 as shown in FIG. 4B stored in the ROM 170 (step S120). The guide display image 320 is an image including an instruction image that prompts the user to adjust the relative position of the projector 100 with respect to the screen SC. In this embodiment, the guide display image 320 is an image including a black background region and an instruction image as shown in FIG. As the instruction image, an image including a white rectangle and an instruction sentence displayed therein can be used as shown in FIG. Here, the directive prompts the user to adjust the relative position of the projector 100 so that the white rectangle is inside the screen.

  After the guide display image 320 is projected on the screen SC, the CPU 120 waits for an input indicating that the adjustment of the relative position of the projector 100 to the screen SC is completed (step S130). This input is performed, for example, when the user presses the adjustment end button 193 of the remote controller 191 or the like. When this input is performed, the CPU 120 performs a captured image generation process (step S200).

  The screen imaging process (step S200) is a process for generating a captured image of the screen SC. Details of the screen imaging process (step S200) will be described later. When the screen imaging process (step S200) ends, the projector 100 performs a boundary line detection process (step S300).

  The boundary line detection process (step S300) is a process for detecting a frame side of the screen SC in the captured image generated by the captured image generation process (step S200) as a boundary line. In the boundary line detection process (step S300), the boundary line detection unit 123 applies an edge extraction filter to the captured image of the screen SC and detects the frame side of the screen as the boundary line. As the edge extraction filter, various filters capable of extracting the contour of the image, such as a differential filter and a Laplacian filter, can be used.

  The corrected shape determination process (step S400) is a process for determining a shape to be corrected for the input image based on the frame sides of the screen SC detected in the boundary line detection process (step S300). In the corrected shape determination process (step S400), the correspondence between the pixel position of the input image and the pixel position of the corrected image whose shape has been corrected is determined. Details of the correction shape determination process will be described later.

  The trapezoidal distortion correction process (step S500) is a process for correcting the input image based on the correction shape determined in the correction shape determination process (step S400) to obtain a corrected image. In the trapezoidal distortion correction process (step S500), the trapezoidal distortion correction unit 136 corrects the corrected image based on the correspondence between the pixel position of the input image and the pixel position of the corrected image obtained in the corrected shape determination process (step S400). The pixel value of is calculated. In general, the pixel position of the input image corresponding to the pixel position of the corrected image is not an integer. Therefore, the pixel value of the corrected image is calculated by performing interpolation using pixel values of several pixels near the pixel position of the input image corresponding to the pixel of the corrected image.

(A3) Captured Image Generation Processing Details of the screen imaging processing (step S200) will be described. The screen imaging process (step S200) is a process for generating a captured image of the screen SC. FIG. 5 shows a flow of processing performed in the screen imaging process (step S200). In this screen imaging process (step S200), the focus blur process (step S210) for adjusting the zoom lens 152 to a state different from the in-focus state, and the screen SC is imaged in a state different from the in-focus state. Imaging processing for generating a captured image (step S220) and adjustment processing for returning the zoom lens 152 to the in-focus state (step S230).

  The focus blurring process (step S210) of the zoom lens 152 is a process of blurring the focus of the zoom lens 152 by changing the state of the zoom lens 152 from a focused state to a state different from the focused state. In the focus blurring process (step S210) of the zoom lens 152, the zoom lens adjustment unit 124 adjusts the zoom lens 152 adjusted to the in-focus state in step S110 to a state different from the in-focus state. Blur the focus. Next, the projector 100 performs an imaging process (step S220).

  In the imaging process (step S220), the imaging unit 180 captures the screen SC while the zoom lens 152 is out of focus, and generates a captured image.

  When the imaging of the screen SC is completed, the zoom lens adjustment unit 124 returns the state of the zoom lens 152 to the focused state (step S230). The zoom lens 152 is adjusted to the in-focus state in step S110, and the state of the zoom lens 152 in the in-focus state is stored in the RAM 160. Therefore, by referring to the RAM 160, the zoom lens adjustment unit 124 can easily return the zoom lens 152 to the in-focus state. When step S230 is performed, the projector 100 performs a boundary line detection process (step S300).

  In the captured image generation process (step S200) of the present embodiment, the focus blurring process (step S210) is performed prior to the imaging process (step S220). The effect when this focus blurring process is performed will be described.

  FIG. 6 shows a screen SC and a projected image projected on the screen SC. Here, it is assumed that the projected image is a black image IMb1. FIG. 6A shows an ideal state where there is no black floating in the projected black image IMb1, and FIG. 6B shows a state where black floating occurs in the projected black image IMb1. ing.

  FIG. 7 is a diagram illustrating a result of applying an edge extraction filter to a captured image obtained by capturing the screen SC and the black image IMb1 illustrated in FIG. FIG. 7A shows the result of applying an edge extraction filter to a captured image obtained by capturing the screen in the state of FIG. On the other hand, FIG. 7B shows the result of applying the edge extraction filter to a captured image obtained by capturing the screen in the state of FIG. The edge strength in FIG. 7 is an output value obtained as a result of applying an edge extraction filter to a captured image. Further, it can be determined that the boundary line exists in a region where the peak of edge strength exists, in other words, a region where the edge strength is larger than other regions and the width is narrow. FIG. 7 shows a one-dimensional change in edge strength along the line B-B ′ shown in FIG. 6.

In an ideal state where there is no black floating in the projected image as shown in FIG. 6A, the edge strength is in the region of the captured image corresponding to the left side S _L and the right side S _R of the screen as shown in FIG. There are peaks. Therefore, since it can be determined that these portions are the left side S _L and the right side S _R of the screen, the frame side of the screen SC in the captured image can be accurately detected. On the other hand, in the state in which black floating has occurred in the projected image as shown in FIG. 6B, not only the area corresponding to the left side S _L and the right side S _R of the screen as shown in FIG. 7B. , there are peaks of the edge intensity in the region corresponding to the left side P _L and right P _R of the projected black image IMb1. Therefore, there is a case where the left side P _L and right P _R of the projected black image IMb1, resulting in erroneously detected as the left S _L and right S _R of the screen.

  FIG. 8 shows the screen SC and the black image IMb2 projected on the screen SC when the focus blurring process according to the present embodiment is performed. FIG. 8A shows an ideal state where there is no black floating in the black image IMb2 as in FIG. 6A, and FIG. 8B shows a state where black floating occurs in the black image IMb2. Is shown. However, compared with FIG. 6B, the black image IMb2 projected in FIG. 8B is out of focus. Therefore, the contour of the black image IMb2 is in a blurred state as compared with the contour of the black image IMb1 in FIG.

  FIG. 9 is a diagram illustrating a result of applying an edge extraction filter to a captured image obtained by capturing the screen SC and the black image IMb2 illustrated in FIG. Similar to FIG. 7, FIG. 9 also shows a one-dimensional change in edge strength along line B-B ′ shown in FIG. 8. FIG. 9A shows the result of applying the edge extraction filter to a captured image obtained by capturing the screen SC in the state of FIG. On the other hand, FIG. 9B shows the result of applying an edge extraction filter to a captured image obtained by capturing the screen SC in the state of FIG. 8B.

In an ideal state where there is no black float in the projected image as shown in FIG. 8A, the edge strength is in the captured image region corresponding to the left side S _L and the right side S _R of the screen as shown in FIG. There are peaks. Accordingly, as in the case of FIG. 7A, the frame side of the screen SC in the captured image can be accurately detected. On the other hand, in the state in which the projected image is black as shown in FIG. 8B, not only the area corresponding to the left side S _L and the right side S _R of the screen as shown in FIG. 9B. , there are peaks of the edge intensity in the region corresponding to the left side P _L and right P _R of the projected black image IMb2. However, compared with the state in which the focus is not blurred as shown in FIG. 7B, the contour of the black image IMb2 is blurred, so that the edge intensity peak height is low and the width is wide. . Therefore, compared to the case of FIG. 7 (B), it is possible to prevent accidentally detects the left P _L and right P _R of the projected black image IMb2 as left S _L and right S _R of the screen Is possible.

(A4) Correction Shape Determination Process Details of the correction shape determination process (step S400) will be described. The corrected shape determination process (step S400) is a process for determining a shape to be corrected for the input image based on the frame sides of the screen SC detected in the boundary line detection process (step S300). FIG. 10 shows the flow of processing performed in the corrected shape determination processing (step S400).

  Note that “panel coordinates” used in the following description are coordinates defined on the light modulation element 130. The panel coordinates can be regarded as equivalent to the coordinates defined on the input image. The “camera coordinates” are coordinates defined on the CCD of the imaging unit 180. The camera coordinates can be regarded as equivalent to the coordinates defined on the captured image.

  In the correction shape determination process (step S400), the trapezoidal distortion correction unit 136 uses the coordinates Pc1 to Pc4 of the four corners of the screen SC in camera coordinates based on the frame edge of the screen detected in the boundary line detection process (step S300). Is detected (step S410). The coordinates Pc1 to Pc4 are coordinates representing the four corner points of the screen SC in the captured image of the screen SC in camera coordinates. The coordinates Pc1 to Pc4 in the captured image are shown in FIG. The coordinates Pc1 to Pc4 represent four straight lines corresponding to the frame sides of the screen detected by the boundary detection process (step S300) as linear functions on the camera coordinates, and obtain the intersection of these linear functions. It is possible to detect.

  When the coordinates Pc1 to Pc4 are acquired, the trapezoidal distortion correction unit 136 projects the measurement image 330 as shown in FIG. 12 from the projector 100 (step S420). As the measurement image 330, for example, an image including a white background region and a predetermined pattern as shown in FIG. 12 can be used. The predetermined pattern included in the measurement image 330 is desirably an image including four or more lattice points in order to calculate a projective transformation equation M1 described later. In this embodiment, the predetermined pattern is composed of two vertical lines 330a and two horizontal lines 330b, and four points that are the intersections of these vertical lines 330a and horizontal lines 330b can be used as lattice points. It is. Therefore, the pattern of the measurement image 330 includes four lattice points. The measurement image 330 corresponds to the measurement image in the claims. When the measurement image 330 is projected, the trapezoidal distortion correction unit 136 causes the imaging unit 180 to capture the projected measurement image 330 and generates a captured image of the measurement image 330 (step S430).

  The trapezoidal distortion correction unit 136 calculates the projective transformation equation M1 based on the captured image of the measurement image 330 (step S440). The projective transformation expression M1 is an expression for associating panel coordinates and camera coordinates. If the coordinates of an arbitrary point Op0 (I0, J0) in the panel coordinates are known, the camera coordinates are set to this point Op0 (I0, J0). The coordinates of the corresponding point Oc0 (i0, j0) can be obtained. The reverse relationship is also established in the same way. Therefore, if the projective transformation formula M1 between the panel coordinates and the camera coordinates is known, the corrected image in which each side of the outer peripheral line of the projected image on the screen SC is parallel to each side of the frame of the screen SC. The shape can be determined. FIG. 13 shows the correspondence between panel coordinates and camera coordinates.

  The correspondence between the panel coordinates represented by the projective transformation formula M1 and the camera coordinates will be described. In general, the transformation formula for projective transformation can be expressed by the following formulas (1) and (2). In the equation, x and y are x and y coordinates in the xy orthogonal coordinate system, X and Y are X and Y coordinates in the XY orthogonal coordinate system, and letters from a to h indicate conversion coefficients. In the two coordinate systems, the origin and the extension direction of the coordinate axes are the same.

  The above formula can be expressed as the following formulas (3) and (4).

  As described above, the measurement image 330 includes four grid points, and the coordinates of these four grid points on the panel coordinates are known. Further, the coordinates of the points on the camera coordinates corresponding to the four grid points are also known from the captured image of the measurement image 330. Accordingly, by substituting the coordinates of these points into the above equations (3) and (4), an eighth order simultaneous equation is obtained, and the conversion coefficients a to h can be obtained.

  For example, among the four lattice points included in the measurement image 330, the coordinates on the panel coordinates of the upper left lattice point Op1 are (I1, J1), and the coordinates of the point Oc1 on the camera coordinates corresponding to the lattice point Op1. Is (i1, j1). At this time, substituting the values of i1, j1, I1 and J1 into the equations (3) and (4), the following two equations are obtained.

  Of the four lattice points included in the measurement image 330, two equations other than the lattice point Op1 are similarly obtained from the combination of one lattice point. Accordingly, eight equations can be obtained from the combination of four lattice points, and the conversion coefficients a to h can be obtained by solving these equations as an eighth-order simultaneous equation.

  When the conversion coefficients a to h are obtained, the coordinates of the points on the camera coordinates corresponding to the arbitrary points whose coordinates are known in the panel coordinates can be obtained from the equations (3) and (4). For example, if the coordinates of the point Op0 (I0, J0) in the panel coordinates are known, the camera coordinates are obtained by substituting I0 and J0 into the equations (3) and (4) and solving them as quadratic simultaneous equations. The coordinates of the point Oc0 (i0, j0) corresponding to the point Op0 (I0, J0) can be obtained.

  Next, the trapezoidal distortion correction unit 136 converts the coordinates Pc1 to Pc4 of the four corners of the screen on the camera coordinates into the corresponding coordinates Pp1 to Pp4 on the panel coordinates using the projective transformation formula M1 obtained in step S440. Conversion is performed (step S450). This projective transformation is shown in FIG. Next, the trapezoidal distortion correction unit 136 calculates a projective transformation formula M2 for causing the coordinates Pp1 to Pp4 corresponding to the four corners of the screen to correspond to the four corner coordinates of the input image on the panel coordinates (step S460). . Here, the projective transformation expression M2 is an expression that associates the four corner coordinates of the input image with the coordinates Pp1 to Pp4 of the four corners of the screen in the panel coordinates. If the four-corner coordinates before conversion (four-corner coordinates of the input image) and the four-corner coordinates after conversion (coordinates Pp1 to Pp4) are known, this projective transformation expression M2 is obtained by the above-described expressions (3) and (4). ) Can be used for calculation. FIG. 15 shows the correspondence between the four corner coordinates of the input image in the panel coordinates and the coordinates Pp1 to Pp4.

  The projective transformation formula M2 obtained in step S460 is a formula that associates the pixel position of the input image with the pixel position of the corrected image whose shape has been corrected. Therefore, by determining the projective transformation equation M2, the correspondence between the pixel position of the input image and the pixel position of the corrected image can be determined. When the projective transformation equation M2 is calculated, the trapezoidal distortion correction unit 136 ends the correction shape determination process (step S400) and performs the trapezoidal distortion correction process (step S500).

  In the trapezoidal distortion correction process (step S500), the trapezoidal distortion correction unit 136 sets a coefficient corresponding to the calculated projective transformation equation M2. The coefficient set here is a coefficient for determining the position of the input image corresponding to the coordinates of the pixel in the corrected image.

  The trapezoidal distortion correction unit 136 uses the set coefficient to determine the position of the input image corresponding to the pixel coordinates in the corrected image. Then, interpolation is performed using the pixel values of several pixels in the vicinity of the determined position of the input image, and the pixel value of the corrected image is calculated. As described above, the trapezoidal distortion correcting unit 136 corrects the input image to generate a corrected image, and outputs the corrected image to the light modulation element driving unit 132.

  The trapezoidal distortion correction unit 136 repeats these processes for each pixel of the corrected image. Thereby, the trapezoidal distortion correction of the projected image is realized.

  As described above, according to the projector 100 of the first embodiment, when the screen SC is imaged by the imaging unit 180, the zoom lens 152 is adjusted to a state different from the in-focus state. When the zoom lens 152 is adjusted to a state different from the in-focus state, the focus of the zoom lens 152 becomes out of focus, so that the projected image projected from the projection optical system 150 as the projection unit has a blurred outline on the projection surface. It becomes a state. For this reason, the outline of the screen SC captured when the zoom lens 152 is out of focus is blurred in comparison with the frame edge of the screen SC captured when the zoom lens 152 is in focus. It becomes the state. Therefore, the projector 100 can suppress erroneous detection of the outline of the black image as the frame side of the screen SC even when the projected black image has a black float. Therefore, the projector 100 can perform appropriate trapezoidal distortion correction processing.

D. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

  (1) In the above embodiment, the projector 100 corrects the input image based on the captured image of the measurement image, but the process for correcting the input image is not limited to this. The projector 100 performs edge extraction processing on the captured image on the projection surface to detect the screen frame side in the captured image, and the relative angle of the projector 100 with respect to the screen SC from the tilt angle of the detected screen frame side in the camera coordinates. It is possible to calculate (vertical and horizontal projection angles) and correct the input image using the calculated relative angles.

  (2) In the above embodiment, when the focus blurring process is performed, the focus adjustment image 310 is used to grasp in advance the state in which the zoom lens 152 is in focus, but the process performed in the focus blurring process is as follows. It is not limited to this. For example, when the projector 100 is within the recommended distance range with respect to the screen SC, a state where the zoom lens 152 is not in focus is stored in the ROM 170, and the zoom lens adjustment unit 124 refers to this stored state. Then, the zoom lens driving unit 155 may be controlled.

  (3) The instruction image included in the guide display image 320 is not limited to characters as display contents, but may be an animation image or a symbol mark indicating a company name. There are no particular restrictions on the content and shape of the guide display.

  (4) In the above embodiment, the projector 100 uses a transmissive liquid crystal light valve as the light modulation element 130, but uses a reflective liquid crystal light valve or DMD (digital mirror device) as the light modulation element. Also good.

  (5) The image sensor used for the imaging unit 180 may be a CMOS instead of the CCD. Furthermore, a semiconductor light source such as an LED or a semiconductor laser may be used instead of the discharge lamp as the light source used in the illumination optical system 140.

  (6) The trapezoidal distortion correction process is started when the user operates the adjustment start button 192 in the remote controller 191, but is not limited thereto. The trapezoidal distortion correction process may be started when the initial screen is projected when the projector 100 is turned on, or an acceleration sensor is installed in the projector 100, the vibration of the projector body is detected, and the installation position adjustment by the user is started. The end may be detected.

  (7) In the embodiment, a part of functions realized by software may be realized by hardware, or a part of functions realized by hardware may be realized by software.

  DESCRIPTION OF SYMBOLS 100 ... Projector, 102 ... Bus, 120 ... CPU, 123 ... Boundary line detection part, 124 ... Zoom lens adjustment part, 125 ... Determination part, 130 ... Light modulation element, 132 ... Light modulation element drive part, 134 ... Image processor DESCRIPTION OF SYMBOLS 136 ... Trapezoid distortion correction part, 140 ... Illumination optical system, 150 ... Projection optical system, 152 ... Zoom lens, 155 ... Zoom lens drive part, 156 ... Zoom adjustment motor, 157 ... Focus adjustment motor, 160 ... RAM, 170 ... ROM, 180 ... imaging unit, 182 ... captured image memory, 190 ... remote control unit, 191 ... remote control, 192 ... adjustment start button, 193 ... adjustment end button, 200 ... cable, 310 ... focus adjustment image, 320 ... guide Display image, 330 ... measurement image, SC ... screen.

Claims (6)

A projection display device that projects and displays an image on a projection surface,
Projection means for projecting the image;
Imaging means for imaging the projection plane and generating a captured image of the projection plane;
Adjusting means for adjusting the projection means to a state different from the in-focus state;
Control means for controlling the adjusting means so as to adjust the projection means to a state different from the in-focus state before the imaging means images the projection plane;
A shape of the projection surface is detected based on a captured image of the projection surface captured when the projection unit is in a state different from the focused state, and the image is matched with the shape of the projection surface. Correction means for correcting;
A projection display device comprising: The projection display device according to claim 1,
The projection means projects an adjustment image,
The imaging means captures the projected adjustment image to generate a captured image of the adjustment image;
The control means determines the in-focus state of the projection means based on the captured image of the adjustment image, and sets the projection means to the in-focus state before the imaging means images the projection plane. A projection display device for controlling the adjusting means to adjust to different states. The projection display device according to claim 2,
The projection display apparatus, wherein the control unit controls the adjustment unit to adjust the projection unit to the in-focus state after the imaging unit has captured the projection plane. The projection display device according to claim 1 or 3,
The projection means projects a measurement image,
The imaging means captures the projected measurement image to generate a captured image of the measurement image,
The projection display device corrects an input image based on a captured image of the projection plane and a captured image of the measurement image. The projection display device according to claim 1 or 3,
The projection display device, wherein the correction unit calculates a relative angle of the projection surface with respect to the projection display device based on a captured image of the projection surface, and corrects an input image based on the relative angle. An image display method for a projection display device that projects and displays an image on a projection surface,
Adjusting the projection means of the projection display device to a state different from the in-focus state;
When the projection means is in a state different from the in-focus state, the projection surface is imaged to generate a captured image,
An image display method for detecting the shape of the projection plane based on the captured image and correcting the image so as to match the shape of the projection plane.

Images