JP2013168922A - Projection type image display device and image projection method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suitably perform automatic correction of trapezoidal distortion of a projected image even in an environment with interference by an external light or in the case of a not pure white object being projected.SOLUTION: A camera unit 104 has an optical axis that is disposed at a position different from an irradiation position of a projection unit 101, and is set to include as much of the irradiation range of the projection unit 101 as possible. The projection unit 101 radiates a specific test pattern, and the test pattern is photographed by the camera unit 104. A correction amount detection unit 105 determines a distance and direction to an object of projection from a photographic image, calculates a projection conversion parameter, and outputs the parameter to an image processing unit 102. After that, every input image is projection-converted by the projection conversion parameter so as to radiate an image with trapezoidal distortion being corrected.

Description

  The technology disclosed in the present specification relates to a projection-type image display apparatus and image projection method for projecting and displaying a playback image of a media, a computer screen, and the like, and a computer program, and in particular, a trapezoid of an image projected on a screen. The present invention relates to a projection type image display apparatus and image projection method for automatically correcting distortion and optical distortion, and a computer program.

  Recently, television reception video, video reproduced from media such as Blu-ray, personal computer (PC) screen, etc. are projected on a large screen using a projection-type image display device, and can be viewed and presented by multiple people. Opportunities are increasing. In addition, small projection type image display devices (pico projectors) that are intended to be used on the palm of the hand or mounted on a portable device have also appeared.

  When an image is projected onto a screen, there is a problem that the image is distorted into a trapezoid because it is projected obliquely onto a projection target (such as a screen wall surface). The trapezoidal distortion automatic correction function projects a test pattern onto the screen, captures an image of the test pattern projected on the screen with the built-in camera, and obtains the positions of the four corners of the screen and the test pattern 4 In general, the three-dimensional information of the screen is obtained based on the corner position to correct the trapezoidal distortion (see, for example, Patent Document 1). If the image projected on the screen is distorted in a trapezoidal shape in the vertical direction, the display image on the image display device is intentionally distorted in the direction opposite to the trapezoidal distortion of the projected image on the screen. The projected image can be a beautiful rectangle.

  However, in order to accurately correct the trapezoidal distortion using the test pattern projected on the screen, it is necessary to capture a clearer projected image of the test pattern with a camera. For this reason, there is a problem in that the trapezoidal distortion cannot be corrected correctly unless it is on a dark room or a pure white screen, and malfunction occurs when it is disturbed by outside light such as natural light. That is, the environment in which the projection type image display apparatus can be used is restricted. For example, the pico projector is small and can be carried anywhere, but if the place of use is limited, the attractiveness is lost.

JP 2007-13810 A

  An object of the technology disclosed in the present specification is to provide an excellent projection image that can appropriately and automatically correct various distortions including a trapezoidal distortion generated in a projection image by using a test pattern projected on a screen. To provide a display device, an image projection method, and a computer program.

  A further object of the technology disclosed in this specification is to automatically correct the trapezoidal distortion of a projected image even in an environment obstructed by external light or when projected onto a non-white projection object. Another object of the present invention is to provide an excellent projection type image display apparatus, image projection method, and computer program.

The present application has been made in consideration of the above problems, and the technology according to claim 1
A projection unit that projects an image onto a projection object;
A camera unit that is installed at a position different from the irradiation position of the projection unit and shoots an image projected on the projection object;
When a test pattern is projected from the projection unit onto the projection target, a background is removed from a test image photographed by the camera unit, and coordinate information relating to the test pattern in the test image after background removal is obtained. A correction amount detection unit that detects and calculates a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
An image correction unit for correcting an image projected from the projection unit based on the correction parameter;
Is a projection type image display apparatus.

  According to the technique described in claim 2 of the present application, the correction amount detection unit of the projection-type image display device according to claim 1 uses a plurality of feature points of the test pattern from the test image after background removal. A coordinate is calculated, a projection transformation parameter for correcting distortion included in the projection image of the subject is calculated from the coordinates of the plurality of feature points, and the image correction unit is configured to calculate an image projected from the projection unit. It is configured to perform projective transformation with transformation parameters.

  According to the technique described in claim 3 of the present application, the camera unit of the projection-type image display device according to claim 1 captures the first test image when the test unit does not irradiate the test pattern. At the same time, a second test image is taken while the test pattern is emitted from the projection unit, and the correction amount detection unit creates a difference between the first test image and the second test image. The background information is removed from the second test image, and the test image after the background removal including the test pattern is obtained.

  According to the technique described in claim 4 of the present application, the camera unit of the projection type image display device according to claim 1 has the first test pattern when the first test pattern is emitted from the projection unit. A test image is taken and a second test image is taken when the second test pattern is emitted from the projection unit, and the correction amount detection unit is configured to capture the first test image and the second test image. A test image obtained by synthesizing the first test pattern and the second test pattern by creating a difference between the first test image and the second test image by removing background information from the first test image and the second test image. It is configured to obtain the test image after removing the background including the pattern.

  According to the technique described in claim 5 of the present application, the correction amount detection unit of the projection type image display device according to any one of claims 3 and 4 includes the first test image and the second test image. After the brightness adjustment is performed, the background information is removed.

  According to the technique described in claim 6 of the present application, the correction amount detection unit of the projection type image display device according to claim 5 uses one of the first test image and the second test image as a reference. The luminance adjustment process is performed by multiplying the pixel value of the other image by the average luminance ratio.

  According to the technique described in claim 7 of the present application, the correction amount detection unit of the projection type image display device according to claim 6 uses one of the first test image and the second test image as a reference. The luminance adjustment process is performed by multiplying the pixel value of the other image by the average luminance ratio.

  According to the technique described in claim 8 of the present application, the correction amount detection unit of the projection type image display apparatus according to claim 1 provides luminance for an area including a test pattern in the test image after background removal. It is configured to normalize.

  According to the technique described in claim 9 of the present application, the correction amount detection unit of the projection type image display device according to claim 8 is a region in which a variance value of pixel values of the test image is equal to or greater than a predetermined threshold. Only configured to normalize luminance.

  According to the technique described in claim 10 of the present application, the projection type image display apparatus according to claim 3 uses a mesh-like test pattern including a plurality of vertical lines and a plurality of horizontal lines. Then, the correction amount detection unit is a projective transformation for correcting distortion based on the coordinates of a plurality of feature points formed by intersections of the vertical line and horizontal line test patterns included in the test image after background removal. The parameters are calculated, and the image correction unit is configured to projectively transform the image projected from the projection unit using the projective transformation parameters.

  According to the technique described in claim 11 of the present application, the test pattern used by the projection type image display device according to claim 10 is two lines corresponding to rectangular diagonal lines that are the outline of the test pattern. It further includes a slit.

  According to the technique described in claim 12 of the present application, the correction amount detection unit of the projection type image display device according to claim 10 is based on the coordinates of the feature points detected from the test image after background removal. The maximum image size that can be projected from the projection unit onto the projection target is estimated.

  According to the technique described in claim 13 of the present application, the projection type image display apparatus according to claim 4 includes a first test pattern including a plurality of vertical lines and a second test including a plurality of horizontal lines.・ Pattern is used. Then, the correction amount detection unit is a projective transformation for correcting distortion based on the coordinates of a plurality of feature points formed by intersections of the vertical line and horizontal line test patterns included in the test image after background removal. The parameters are calculated, and the image correction unit is configured to projectively transform the image projected from the projection unit using the projective transformation parameters.

  According to the technique described in claim 14 of the present application, the correction amount detection unit of the projection type image display device according to claim 13 is based on the coordinates of the feature points detected from the test image after background removal. The maximum image size that can be projected from the projection unit onto the projection target is estimated.

  According to the technique described in claim 15 of the present application, the correction amount detection unit of the projection type image display apparatus according to claim 1 further includes noise caused by autocorrelation with respect to the test image after the background information is removed. After performing the reduction process, the coordinate information of each feature point related to the test pattern in the test image is detected, and the correction parameter is calculated based on the coordinate information of each feature point. ing.

  According to the technique of claim 16 of the present application, the correction amount detection unit of the projection type image display apparatus according to claim 15 includes the test pattern including a test pattern of a plurality of vertical lines or a plurality of horizontal lines. After the image is subjected to an angle search in the vertical or horizontal direction and noise reduction processing is performed, the coordinates of a plurality of feature points consisting of the intersections of the test patterns of the vertical and horizontal lines are calculated, and the coordinates of the coordinates of the intersections are calculated. A projective transformation parameter for correcting the distortion based on the information is calculated.

  According to the technique described in claim 17 of the present application, the correction amount detection unit of the projection type image display apparatus according to claim 16 converts each test pattern of vertical lines or horizontal lines included in the test image to a secondary order. It is configured to detect as a curve.

  According to the technique described in claim 18 of the present application, the correction amount detection unit of the projection type image display apparatus according to claim 16 is configured so that the angle is centered on a result of an adjacent line segment that has just been subjected to an angle search. It is configured to search.

Moreover, the technology described in claim 19 of the present application is:
Projecting a test pattern onto the projection object;
Capturing a test image by capturing a test pattern projected on the projection target; and
Removing a background from the test image;
Detecting coordinate information related to the test pattern included in the test image after removing the background, and calculating correction parameters for correcting the image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
Is an image projection method.

In addition, the technique described in claim 20 of the present application is:
A procedure for projecting a test pattern onto a projection object;
A procedure for acquiring a test image by photographing a test pattern projected on a projection object,
Removing a background from the test image;
A procedure for detecting coordinate information related to the test pattern included in the test image after removing the background and calculating a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
Is a computer program written in a computer-readable format to cause a computer to execute.

  The computer program according to claim 20 of the present application defines a computer program written in a computer-readable format so as to realize predetermined processing on a computer. In other words, by installing the computer program according to claim 20 of the present application on a computer, a cooperative operation is exhibited on the computer, and the same effect as the image projection method according to claim 19 of the present application is obtained. be able to.

  According to the technique disclosed in this specification, an excellent projection image that can suitably automatically correct various distortions including a trapezoidal distortion generated in a projection image using a test pattern projected on a screen. A display device, an image projection method, and a computer program can be provided.

  Further, according to the technology disclosed in this specification, it is possible to appropriately automatically correct the trapezoidal distortion of a projected image even in an environment obstructed by external light or when projected onto a non-white projection object. An excellent projection-type image display device, image projection method, and computer program that can be provided can be provided.

  Other objects, features, and advantages of the technology disclosed in the present specification will become apparent from a more detailed description based on the embodiments to be described later and the accompanying drawings.

FIG. 1 is a diagram schematically illustrating a configuration of a projection image display apparatus 100 according to an embodiment of the technology disclosed in this specification. FIG. 2 is a diagram illustrating an internal configuration example of the projection unit 101. FIG. 3 is a diagram illustrating an internal configuration example of the image processing unit 102. FIG. 4 is a diagram illustrating an internal configuration example of the correction amount detection unit 105. FIG. 5A is a diagram showing an example of a test pattern used for calculating a projective transformation parameter for correcting distortion of a projected image. FIG. 5B is a diagram showing another example of a test pattern used for calculating a projective transformation parameter for correcting distortion of a projected image. FIG. 5C is a diagram showing still another example of the test pattern used for calculating the projective transformation parameter for correcting the distortion of the projection image. FIG. 5D is a diagram for explaining a process of calculating projective transformation parameters using the test pattern shown in FIG. 5C. FIG. 5E is a diagram showing another example of the first test pattern used for calculating the projective transformation parameter for correcting the distortion of the projected image. FIG. 5F is a diagram showing another example of the second test pattern used for calculating the projective transformation parameter for correcting the distortion of the projected image. FIG. 5G is a diagram showing a modification of the test pattern shown in FIG. 5C. FIG. 5H is a diagram showing a modification of the first test pattern shown in FIG. 5E. FIG. 5I is a diagram showing a modification of the second test pattern shown in FIG. 5F. FIG. 5J shows how the average luminance A_Area (0, 1) is obtained in the area Area (0, 1) of the first test image A photographed by irradiating the first test pattern shown in FIG. 5H. FIG. FIG. 5K shows how the average luminance B_Area (0, 1) is obtained in the area Area (0, 1) of the first test image B photographed by irradiating the second test pattern shown in FIG. 5H. FIG. FIG. 6 is a flowchart showing a processing procedure for correcting the trapezoidal distortion of the projected image on the projection object in the projection type image display apparatus 100. FIG. 7A is a diagram exemplifying a first test image A photographed when a first test pattern composed of two horizontal and vertical lines is irradiated onto a projection target. FIG. 7B is a diagram illustrating a second test image B photographed when the projection target is irradiated with a second test pattern composed of two vertical and horizontal lines. FIG. 8A is a diagram illustrating a background removed image C obtained by subtracting the second test image B from the first test image A. FIG. FIG. 8B is a diagram showing a background removed image D obtained by subtracting the first test image A from the second test image B. FIG. 9A is a diagram for explaining processing for performing angle search and noise reduction on a background-removed image. FIG. 9B is a diagram for explaining processing for performing angle search and noise reduction on the background-removed image. FIG. 9C is a diagram for explaining processing for performing angle search and noise reduction on the background-removed image. FIG. 9D is a diagram for explaining processing for performing angle search and noise reduction on the background-removed image. FIG. 10A is a diagram showing a background-removed image C2 obtained by performing a powerful noise reduction process based on autocorrelation on the background-removed image C. FIG. 10B is a diagram showing a background removed image D2 obtained by performing a powerful noise reduction process by autocorrelation on the background removed image D. FIG. 11A is a diagram for explaining a process of detecting intersections by detecting four line segments included in the background-removed images C2 and D2 as quadratic curves. FIG. 11B is a diagram for explaining a process of detecting intersections by detecting four line segments included in the background-removed images C2 and D2 as quadratic curves. FIG. 11C is a diagram for explaining a process of detecting intersections by detecting four line segments included in the background-removed images C2 and D2 as quadratic curves. FIG. 12 is a diagram illustrating a state where pincushion distortion occurs in an image projected onto a subject. FIG. 13 is a diagram illustrating a state in which barrel distortion has occurred in a projected image on a subject.

  Hereinafter, embodiments of the technology disclosed in this specification will be described in detail with reference to the drawings.

  FIG. 1 schematically shows a configuration of a projection type image display apparatus 100 according to an embodiment of the technology disclosed in this specification. The illustrated projection type image display apparatus 100 includes a projection unit 101, an image processing unit 102, an image input unit 103, a camera unit 104, and a correction amount detection unit 105. Hereinafter, each part will be described.

  The image input unit 103 inputs an image signal from a projection image supply source such as a personal computer, a TV receiver, or a Blu-ray disc playback device (none of which is shown).

  The image processing unit 102 processes an image output from the projection unit 101. The image output from the image processing unit 102 is an external image supplied from the image input unit 103 and a test pattern generated in the image processing unit 102. In the image processing unit 102, the distortion of the projected image is also corrected based on the correction parameter supplied from the correction amount detection unit 105. The distortion to be corrected includes not only the trapezoidal distortion based on the three-dimensional positional relationship between the projection unit 101 and the projection target but also the optical distortion caused by the optical system of the projection unit 101 and the camera unit 104.

  The projection unit 101 projects the image output from the image processing unit 102 onto a projection target such as a screen (not shown). By projecting from the projection unit 101 onto the subject (screen wall surface) from an oblique direction, trapezoidal distortion occurs in the projected image.

  The camera unit 104 captures a test pattern projected from the projection unit 101 onto the projection target. The correction amount detection unit 105 calculates a correction amount for correcting the trapezoidal distortion and the optical distortion included in the projection image from the projection unit 101 using the test pattern image captured by the camera 104 to obtain an image. The data is output to the processing unit 102. By performing projective transformation on the output image, trapezoidal distortion and optical distortion can be corrected. In the present embodiment, the correction amount detection unit 105 calculates a projective transformation parameter as the correction amount.

  In the present embodiment, the camera unit 104 is disposed at a position different from the irradiation position of the projection unit 101, and the optical axis is set so that the imaging range includes the irradiation range of the projection unit 101 as much as possible. When a specific test pattern is emitted from the projection unit 101, the camera unit 104 captures an image. Then, the correction amount detection unit 105 obtains the distance and direction to the projection object from the captured image, calculates the projective transformation parameter, and outputs it to the image processing unit 102. Thereafter, the image processing unit 102 performs projection conversion on all images input from the image input unit 103 using the projective conversion parameters, and the projection unit 101 emits an image with trapezoidal distortion and optical distortion corrected.

  FIG. 2 shows an internal configuration example of the projection unit 101. The illustrated projection unit 101 includes a liquid crystal panel 201, an illumination optical unit 202, a liquid crystal panel drive unit 204, and a projection optical unit 203.

  The liquid crystal panel driving unit 204 drives the liquid crystal panel 201 based on the image signal input from the image processing unit 102 and draws a projected video on the display screen. The illumination optical unit 202 irradiates the liquid crystal panel 201 from the back side. When the projection image display apparatus 100 is a pico projector, for example, an LED (Light Emitting Diode) or a laser is used as the light source of the illumination optical unit 202. The projection optical unit 203 enlarges and projects the light transmitted through the liquid crystal panel 201 onto a projection target (not shown). From the projection unit 101, an input image to the image input unit 103 or a test pattern generated in the projection type image display apparatus 100 is projected. The projection optical unit 203 includes one or more optical lenses. It is assumed that the projection optical unit 203 has lens distortion. For this reason, the projection image has optical distortion in addition to trapezoidal distortion.

  FIG. 3 shows an internal configuration example of the image processing unit 102. The illustrated image processing unit 102 includes an image writing / reading control unit 301, a frame memory 302, an image correction unit 303, an image quality adjustment unit 304, a test pattern generation unit 305, and an output image switching unit 305. Yes.

  An image supplied from the image input unit 103 is stored in the frame memory 302. An image writing / reading control unit 301 controls writing and reading of an image frame to / from the frame memory 302.

  The image correction unit 303 performs projective conversion on the image read from the frame memory 302 based on the projection conversion parameter received from the correction amount detection unit 105, and the trapezoidal distortion is eliminated when the image is projected from the projection unit 101 onto the subject. Make corrections.

  The image quality adjustment unit 304 performs image quality adjustment such as luminance, contrast, synchronization, tracking, color density, and hue so that the projected image after distortion correction is in a desired display state.

  The test pattern generation unit 305 generates a test pattern used when the correction amount detection unit 105 calculates projective transformation parameters. The test pattern has a geometric shape that makes it easy to obtain three-dimensional information of a screen that is a projection target. What test pattern is used will be described later.

  The output image switching unit 306 switches the image output to the projection unit 101. For example, when a presentation is performed by projecting an input image from an image supply source such as a personal computer, a TV receiver, or a media playback device (all not shown) onto the projection object, the output image switching unit 306 has an image quality. An output image from the correction unit 304 is output to the projection unit 101. When calculating the projective transformation parameters for correcting the trapezoidal distortion and optical distortion of the projected image, the output image switching unit 306 outputs the test pattern generated by the test pattern generating unit 305 to the projecting unit 101.

  FIG. 4 shows an internal configuration example of the correction amount detection unit 105. The illustrated correction amount detection unit 105 includes a captured image writing / reading control unit 401, a captured image memory 402, a feature point calculation unit 403, and a projective transformation parameter calculation unit 404.

  The captured image memory 402 stores captured images of the camera unit 104. In the present embodiment, the captured image memory 402 has a size that can store at least two frames of captured images of the camera unit 104.

  A captured image writing / reading control unit 401 controls writing and reading of a captured image to / from the captured image memory 402.

  The feature point calculation unit 403 uses the captured image read from the captured image memory 402 to obtain the coordinates of feature points such as the four corners of the test pattern included in the captured image. Then, the projective transformation parameter calculation unit 404 obtains the distance and direction from the projection unit 101 to the projection object based on the calculated coordinates of the feature points, and trapezoidal distortion or optical distortion of the image projected on the projection object. Calculate the projective transformation parameters to correct

  In order to calculate the correction amount of the trapezoidal distortion and the optical distortion using the test pattern projected on the projection target, it is necessary to extract clearer information on the test pattern. For this reason, unless it is on a dark room or a pure white screen, it may be disturbed by outside light such as natural light and malfunction. For example, when the projection-type image display device 100 is a pico projector, in order to obtain the advantage that it can be carried anywhere and used, the test pattern information must be accurately extracted even on a patterned screen that is exposed to external light. Don't be. On the other hand, in the present embodiment, when extracting test pattern information from the captured image of the camera unit 104, background information is removed, and thus, under an environment obstructed by external light, Even when projected onto a non-white projection object, automatic correction of trapezoidal distortion and optical distortion is realized.

  In this embodiment, the camera unit 104 performs two shootings in order to remove background information. For example, the first test image shot without irradiating the test pattern with the same background and the second test image shot with the test pattern irradiated are stored in the captured image memory 402. Alternatively, a first test image photographed when the first test pattern is irradiated and a second test image photographed when the second test pattern different from the first test pattern is irradiated. Stored in the captured image memory 402. Then, the feature point calculation unit 403 creates a difference between the first test image and the second test image read from the captured image memory 402, removes the background information, and performs the first test pattern and the second test image. The test pattern information obtained by combining the test patterns is clarified. As a result, the coordinates of the feature points such as the four corners of the test pattern are accurate even in an environment obstructed by external light or when the test pattern is projected onto a non-white (patterned) projection object. Thus, it is possible to perform projective transformation parameters for correcting trapezoidal distortion and optical distortion without malfunction.

  Here, in order for the feature point calculation unit 403 to extract the coordinates of feature points such as the four corners of the test pattern, the test pattern generated by the test pattern generation unit 305 includes two horizontal lines and two vertical lines. It preferably includes a line.

  For example, as shown in FIG. 5A, when using a test pattern that is a rectangle substantially along the periphery of the irradiation range of the projection unit 101, the first test image is taken without irradiating the test pattern, and the test is performed.・ Take a second test image by irradiating the pattern, make a difference between the first test image and the second test image, and remove the background information to make the test pattern information clearer Can do.

  Further, as shown in the left of FIG. 5B, the first test pattern including two horizontal lines along the upper and lower edges of the irradiation range of the projection unit 101, and the projection unit 101 as shown in the right of FIG. 5B. A second test pattern including two vertical lines along the left and right edges of the illumination range may be used. In this case, the first test pattern is irradiated to shoot the first test image, and the second test pattern is irradiated to shoot the second test image, and the first test image and the second test image are shot. If the difference between the test images is made and the background information is removed, similarly, the test pattern that becomes a rectangle substantially along the periphery of the irradiation range of the projection unit 101 can be sharpened. The test image after background removal includes a rectangular test pattern obtained by synthesizing the first test pattern and the second test pattern (that is, similar to the test pattern shown in FIG. 5A). Then, the projection transformation parameter calculation unit 404 obtains the coordinates of the feature points at the four corners of the rectangular test pattern, and obtains the distance and direction from the projection unit 101 to the projection object based on the coordinate information, Projection transformation parameters for correcting trapezoidal distortion of an image projected on the projection target can be calculated.

  The display image of the liquid crystal panel 201 including the test pattern is, for example, 630 × 360 pixels, and the widths of the horizontal lines and the vertical lines arranged in the peripheral portion are, for example, 6 pixels.

  If only the coordinates of the four corner points of the irradiation range of the projection unit 101 are extracted as feature points, the trapezoidal distortion can be corrected. If not only the coordinates of the four corner points of the irradiation range of the projection unit 101 but also the coordinates of more feature points can be extracted from the test pattern, not only the trapezoidal distortion of the entire projection image but also locally. More detailed distortion correction such as the generated distortion can be performed. Therefore, the test pattern (or the combination of the first test pattern and the second test pattern) is not only two horizontal lines and two vertical lines, but also three or more horizontal lines and three or more vertical lines. You may comprise by the combination of a line.

  For example, as shown in FIG. 5C, when using a test pattern composed of a combination of three or more horizontal lines and three or more vertical lines, the first test image is taken without irradiating the test pattern. If you irradiate the test pattern and take the second test image, make a difference between the first test image and the second test image, and remove the background information, more than three horizontal lines and three The information on the mesh-like test pattern where the above vertical lines intersect can be clarified.

  Further, a first test pattern composed of three or more horizontal lines as shown in FIG. 5E and a second test pattern composed of three or more vertical lines as shown in FIG. 5F may be used. Good. In this case, the first test pattern is irradiated to shoot the first test image, and the second test pattern is irradiated to shoot the second test image, and the first test image and the second test image are shot. If the difference between the test images is made and the background information is removed, the test pattern information can be similarly clarified. The test image after background removal includes a mesh-like test pattern obtained by synthesizing the first test pattern and the second test pattern (that is, similar to the test pattern shown in FIG. 5C). .

  Here, due to circumstances such as the area of the projection target being smaller than the irradiation range of the projection unit 101 (or the area of the flat part is narrow), a part of the test pattern projected from the projection unit 101 is missing. May end up. In FIG. 5D, a missing region of the mesh-shaped test pattern shown in FIG. 5C (or a combination of the test patterns shown in FIGS. 5E and 5F) is indicated by a dotted line. Thus, when the entire mesh-like test pattern cannot be used for the calculation of the projective transformation parameter, the feature point calculation unit 403 has the largest rectangular 4 that can be extracted from the test pattern normally projected on the projection target. The coordinates of the corners are calculated, and the projection transformation parameter calculation unit 404 may calculate the projection transformation parameters using the coordinates of the four corners of the largest rectangle.

  The feature point calculation unit 403 obtains four corners (four intersections indicated by X in the drawing) of the largest rectangle (region drawn with diagonal lines in the drawing) extracted from the test pattern projected normally in FIG. 5D. calculate. Then, the projection transformation parameter calculation unit 404 may use the four corner points to calculate the projection transformation parameters for correcting the trapezoidal distortion.

  FIG. 5G shows a modification of the test pattern shown in FIG. 5C. Taking the first test image without irradiating the test pattern, taking the second test image by irradiating the test pattern, creating a difference between the first test image and the second test image, If the background information is removed, the information on the fine mesh test pattern can be clarified. 5H shows a modification of the first test pattern shown in FIG. 5E, and FIG. 5I shows a modification of the second test pattern shown in FIG. 5F. In this case, the first test pattern is irradiated to shoot the first test image, and the second test pattern is irradiated to shoot the second test image, and the first test image and the second test image are shot. If the difference between the test images is made and the background information is removed, the test pattern information can be similarly clarified. The test image after background removal includes a mesh-like test pattern obtained by synthesizing the first test pattern and the second test pattern (that is, similar to the test pattern shown in FIG. 5G). .

  As shown in FIGS. 5G to 5I, when the test pattern is made into a finer mesh, the area of the projection target is smaller than the irradiation range of the projection unit 101 (or the area of the flat part is narrow). Accordingly, when a part of the test pattern projected from the projection unit 101 is missing, the maximum rectangle that can be extracted from the test pattern can be estimated with high accuracy. The feature point calculation unit 403 may calculate projective transformation parameters using the estimated feature points at the four corners of the maximum rectangle. In addition, by detecting feature points from the test image after background removal, it is possible to easily detect how much of the test pattern has been irradiated on the screen, so the maximum image size that can be irradiated to the screen is also estimated. It becomes possible. The image correcting unit 303 may perform distortion correction of the projected image using the projective transformation parameter and adjust the size of the projected image according to the maximum image size.

  Each of the test patterns shown in FIGS. 5G to 5I includes a mesh or two slits that divide the horizontal and vertical test patterns corresponding to a rectangular diagonal line that is the outline of the test pattern. It is out. The intersection of the slits corresponds to approximately the center of the test pattern. Therefore, it is possible to easily obtain the position of the maximum range of the test pattern irradiated on the screen (up to the upper and lower and left and right intersections can be detected) on the basis of the intersection of the slits.

  As described so far, in the present embodiment, the difference between the first test image and the second test image is taken to remove the background to obtain a clearer test pattern. However, in a system in which the user cannot determine the influence of external light or the exposure of shooting (when the camera unit 104 cannot be controlled or the camera unit 104 is not released to the user), the first test The brightness may differ greatly between the image and the second test image. If there is a luminance difference between the captured test images, only the background cannot be removed with high accuracy even if the difference is taken.

  Therefore, the feature point calculation unit 403 adjusts the luminance between the first test image and the second test image, and then performs background removal by taking the difference. For example, the process of multiplying the ratio of the average luminance by the pixel value of the other test image using either one of the test images as a reference. In addition, the luminance adjustment is performed locally in consideration of a local luminance difference, not a uniform luminance difference over the entire captured image.

  Specifically, the average luminance is obtained for each region obtained by dividing each test image into a plurality of horizontal and vertical directions. Then, in the corresponding areas of each test image, the ratio of the average luminance based on one is calculated and multiplied by the other pixel value.

  In FIG. 5J, the first test image A photographed by irradiating the first test pattern shown in FIG. 5H is divided into a plurality of parts in the horizontal and vertical directions, and the area Area (0, 0, 0th row, first column) is divided. 1) shows how the average luminance A_Area (0, 1) is obtained. Similarly, in FIG. 5K, the average luminance B_Area (0) in the area Area (0, 1) in the 0th row and the first column of the second test image B photographed by irradiating the second test pattern shown in FIG. 0, 1) is shown.

  Then, when the average brightness ratio Scale based on the first test image A is calculated according to the following formula (1), the average ratio Scale is the same as that of the second test image B as shown in the following formula (2). A second test image B ′ after luminance adjustment is obtained by multiplying the pixel value of each pixel in the area Area (0, 1).

  The feature point calculator 403 normalizes the luminance of the image after removing the background by taking the difference between the first test image and the second test image. However, when the projection object is in a bright environment, the brightness of the test pattern that can be photographed by the camera unit 104 is lowered. If the luminance of the background-removed image is normalized in accordance with this, there is a problem that noise in a region unrelated to the test pattern is amplified and detection accuracy is deteriorated. Therefore, in the background removal image, the test pattern area including the test pattern is distinguished from the other areas, and only the area including the former test pattern is normalized to prevent noise amplification. Yes.

  A method for determining whether or not each region is a test pattern region is arbitrary. For example, attention may be paid to the fact that the pixel value changes abruptly at the edge portion of the test pattern and the variance of the pixel value becomes high, and it may be determined whether the test pattern region is based on the variance value. That is, the feature point calculation unit 403 acquires a variance value for each region, and normalizes the luminance if the variance value is equal to or greater than the threshold value, but does not normalize the region where the variance value is smaller than the threshold value. To do.

  Alternatively, focusing on the fact that the portion irradiated with the test pattern has high luminance, it may be determined whether or not it is a test pattern region based on the average luminance. That is, the feature point calculation unit 403 obtains the average luminance for each region, and normalizes the luminance if the average luminance is equal to or greater than the threshold, but does not normalize the region where the average luminance is less than the threshold. To do. However, in the average luminance threshold processing, if the test pattern irradiation from the projection unit 101 and the shutter timing of the camera unit 104 are not synchronized, there is a concern that the low luminance region of the test pattern may be erroneously determined. It is necessary to pay attention to.

  In addition, the feature point calculation unit 403 detects two intersecting line segments as straight lines or quadratic curves in order to obtain the coordinates of feature points such as the four corner points from the image after background removal (that is, The equation of the line segment is determined) and the method of calculating the intersection of the two line segments is used.

  As described above, background information is removed from the first test image and the second test image for test pattern calculation. However, due to the influence of external light such as natural light, it is assumed that the image after removing the background situation is still an image with a very low contrast of the test pattern. Therefore, the feature point calculation unit 403 performs powerful noise reduction processing on the image from which the background information is removed. For example, a powerful noise reduction process by autocorrelation of line segments is performed on an image from which background information has been removed. In other words, when calculating the line segment equation, the line segment constituting the test pattern and other noise components are separated by detecting the tilt angle of the line segment and averaging the pixel information in that direction. can do.

  FIG. 6 shows, in the form of a flowchart, a processing procedure for correcting distortion of the projection image on the projection object in the projection type image display apparatus 100. The illustrated processing procedure includes the following processing steps.

S601: Test image irradiation S602: Test image capture and storage S603: Calculate the coordinates of a plurality of feature points in the test pattern S604: Calculate the position of the projection object from the coordinates of the plurality of feature points in the test pattern Calculate direction S605: Calculate projective transformation parameters based on the position and direction of the projection object S606: Correct the trapezoidal distortion of the projected image on the projection object by projective transformation

  Hereinafter, each processing step will be described in detail.

  First, a test image including a test pattern generated by the test pattern generation unit 305 is irradiated from the projection unit 101 to the projection target (step S601). Then, the camera unit 104 captures the test image projected on the projection target and stores it in the captured image memory 402 (step S602).

  As described above, in this embodiment, in order to remove the background information from the test image, two shootings are performed in steps S601 and S602. In the following description, it is assumed that the first test image and the second test image captured when each of the first test pattern and the second test pattern different from each other are irradiated are stored in the captured image memory 402. To do.

  In FIG. 7A, the first test pattern consisting of two horizontal and vertical lines along the left and right edges of the irradiation range is photographed by the camera unit 104 when the projection unit 101 irradiates the projection target. One test image A is illustrated. In FIG. 7B, a second test pattern consisting of two vertical and horizontal lines along the upper and lower edges of the irradiation range is photographed by the camera unit 104 when the projection unit 101 irradiates the projection target. The second test image B is illustrated.

  Next, the feature point calculation unit 403 uses the first test image A and the second test image B read from the photographed image memory 402 and projects them as feature points of the test pattern included in these photographed images. The coordinates of the four corners of the rectangle obtained by combining the first test pattern and the second test pattern are obtained (step S603).

  Here, the feature point calculation unit 403 creates a difference between the first test image A and the second test image B, removes the background information, and sharpens the test pattern information. Further, it is assumed that the background information is removed after adjusting the luminance between the first test image A and the second test image B (described above).

  FIG. 8A shows a background removed image C obtained by subtracting the second test image B from the first test image A. In this subtraction process, the second test image B is subtracted from the first test image A in units of pixels, and the pixel value is set to 0 for pixels in which the difference is negative. As a result, as shown in FIG. 8A, in the background removal image C, the background disappears from the first test image A, and the first test consisting of two vertical lines along the left and right edges of the irradiation range.・ Pattern remains.

  FIG. 8B shows a background removal image D obtained by subtracting the first test image A from the second test image B. In this subtraction process, the first test image A is subtracted from the second test image B in units of pixels, and the pixel value is set to 0 for pixels in which the difference is negative. As a result, as shown in FIG. 8B, in the background removal image D, the background disappears from the second test image B, and the second test image composed of two horizontal lines along the upper and lower edges of the irradiation range. The pattern is left behind.

  However, in FIG. 8A and FIG. 8B, the feature point calculation unit 403 obtains the maximum luminance value for each of the four line segments included in the background-removed images C and D, and performs the luminance normalization process (described above). This is the result.

  When test images A and B irradiated in a dark atmosphere are photographed, clear background-removed images C and D can be obtained. However, when the test images A and B are photographed in an atmosphere irradiated with a little external light such as natural light, only the background-removed images C and D with low contrast can be obtained.

  Therefore, the feature point calculation unit 403 performs powerful noise reduction processing based on the autocorrelation of the line segment on the image from which the background information is removed.

  A wide vertical line has a strong vertical correlation even if it is slightly inclined (similarly, a wide horizontal line has a strong horizontal correlation). Therefore, for a certain pixel line, if the autocorrelation is calculated by calculating the sum of the pixel values of the same horizontal pixel position in the range of about ± 5 lines above and below, at the pixel position in the vicinity where the vertical line passes The pixel value is amplified about 10 times. On the other hand, since there is only random noise at the pixel position where the vertical line does not pass, even if the sum of the pixel values for ± 5 lines is taken, it is amplified only about 3 times. The effect of noise reduction using such autocorrelation is about three times.

  If the number of pixel lines for autocorrelation is increased, the effect of noise reduction is further improved. For example, when the sum of the pixel values at the same horizontal pixel position is calculated over 100 pixel lines, the pixel value is amplified about 100 times at the pixel position near the vertical line. However, in order to obtain autocorrelation for 100 pixel lines, the angle error of the line segments must be within 0.5 degrees.

  Therefore, the effect of noise reduction is obtained more positively by combining a technique called angle search.

  For example, the vertical lines on the left and right edges, which are the test pattern shown on the right in FIG. 5B, may be inclined up to 13 degrees. Therefore, first, an angle having the highest autocorrelation at 1 degree intervals is roughly searched within a range of ± 13 degrees. Next, a detailed angle search by autocorrelation is performed within a range of 2 degrees near the angle at which the autocorrelation is maximized by the rough angle search. If the angle search is repeated three times while reducing the angle search range and the search interval in this way, the error is within 0.07 degrees.

  If the angle of the vertical line that becomes the test pattern can be accurately detected through the noise reduction process in this way, the sum of the pixel values for 100 pixel lines in the actual direction of the angle is obtained and the autocorrelation is obtained. Take. Also, normalization is performed by dividing the sum so that the maximum value becomes 255. Actually, since the pixel values are summed in the 191 pixel line, the effect of noise reduction is about 14 times.

  9A to 9D illustrate how the angle search and noise reduction processing are performed on the background-removed image C shown in FIG. 8A.

  In the background-removed image C, two vertical lines are on the left and right of the screen half. Therefore, in the following, a process for the left half of the screen will be described by dividing the screen into left and right as shown in FIG. 9A. It should be understood that the processing for the right half of the screen is the same.

(1) First, the angle is searched while moving in the horizontal direction pixel by pixel on the center line v / 2 of the vertical size v. As shown in FIG. 9B, first, an angle range of ± 13 degrees (a range between the dotted lines in the figure) is searched at intervals of 1 degree, and pixel values for upper and lower n pixels are added.

(2) Then, as a result of the angle search for the interval of ± 13 degrees, on the center line 2 / v of the vertical size v, the angle (anmax01) and the pixel position (imax, v / 2) is determined.

(3) Subsequently, as shown in FIG. 9C, with respect to the determined angle (anmax01), the angle range of (anmax01-1), (anmax01 + 1) (the range between the dotted lines in the figure) is 1 / A second angle search is performed at intervals of 4 degrees.

(4) Then, at the angle (anmax02) at which the autocorrelation is maximized in the second angle search, and (anmax02-1 / 4) and (anmax02 + 1/4) with respect to (anmax02), A third angle search is performed at 1/16 degree intervals. Thus, by repeating the angle search three times, the error is within 0.07 degrees. With the above processing, the angle (anmax) is determined for the vertical line in the left half of the screen.

(5) Subsequently, as shown in FIG. 9D, the position x in the horizontal direction (x direction) with respect to the range of y = (m / 2) to v− (m / 2) in the vertical direction (y direction). = (Y−v / 2) × tan (anmax) + imax centered within a range of ± 15 pixels in the horizontal direction (range between the dotted lines in the figure), and pixels for m pixels are averaged in the vertical direction , Which is the pixel value at position (x, y). As a result, an image subjected to the noise reduction process is completed within a range surrounded by a dotted line in FIG. 9D.

  By applying the above processes (1) to (5) also to the right half of the screen, a powerful noise reduction process based on autocorrelation is realized for the background-removed image C. The powerful noise reduction process based on the autocorrelation can be similarly applied to the background-removed image D including two horizontal lines shown in FIG. 8B.

  FIG. 10A shows a background removed image C2 obtained by performing a powerful noise reduction process by autocorrelation on the background removed image C shown in FIG. 8A. FIG. 10B shows a background removed image D2 obtained by performing a powerful noise reduction process by autocorrelation on the background removed image D shown in FIG. 8B.

  The feature point calculation unit 403 calculates the intersection of two horizontal lines and two vertical lines included in the background removal images C2 and D2, and coordinates of the four corners of the four corners of the irradiation range of the projection unit 101 Ask for.

  Here, if the distortion generated in the projected image is a trapezoidal distortion, the projected image of the test pattern composed of a combination of vertical lines and horizontal lines should be linear. However, since the projection optical unit 203 and the camera unit 104 have lens distortion, these four line segments projected on the projection target (in other words, the line segments observed in the background-removed images C2 and D2) are slight. It is a curve.

  Therefore, the feature point calculation unit 403 detects these four line segments included in the background-removed images C2 and D2 as quadratic curves so as to obtain intersection points.

  Specifically, the feature point calculation unit 403 examines the luminance values of the background-removed images C2 and D2 in a direction substantially perpendicular from the outside of the screen toward the center, and among these, the center of the region with the highest luminance value Thus, the position data of the line segment is obtained. FIG. 11A shows a state in which line segment position data is extracted from the background-removed image. In the figure, the position data of the line segment is drawn with a solid line.

Subsequently, the feature point calculation unit 403 calculates coefficients a, b, and c of a quadratic curve y = ax ₂ + bx + c that approximates each of the four line segments from the plurality of position data using, for example, the least square method. To do. By using the method of least squares for detecting a quadratic curve and comprehensively handling many points of the line segment, it is possible to obtain accuracy of a fraction of one pixel. FIG. 11B shows a state in which a quadratic curve is detected from two horizontal lines and vertical lines in the background-removed images C2 and D2. In the figure, a quadratic curve detected from two horizontal lines and a vertical line is drawn with dotted lines.

  And the feature point calculation part 403 calculates | requires the intersection of four quadratic curves, and makes this a coordinate of four corners. FIG. 11C shows how to obtain the intersection of four quadratic curves. In the figure, the obtained positions of the four corners are indicated by X.

  If the distortion that occurs in the projected image is trapezoidal, the projected image of the test pattern consisting of a combination of vertical and horizontal lines should be linear (as described above), with an angle of 0 degrees in the vertical and horizontal directions. By searching, the line segment of the test pattern can be detected. However, since there is actually lens distortion in the projection optical unit 203 and the camera unit 104, these test patterns projected on the projection target have nonlinear distortion, so that the search range is centered on the vertical and horizontal directions. In some cases, there is no optimal solution. In general, a pincushion distortion (see FIG. 12) in which the projected image is reduced at the center of the field of view and enlarged as it goes to the end due to lens distortion, or barrel distortion that is enlarged at the center of the field of view and shrinks toward the end (See FIG. 13) is known to occur. Both the pincushion type and barrel type distortions have symmetrical distortion amounts on the left and right and top and bottom.

  If an angle search is performed by expanding the search range, an optimal solution can be found, but the calculation amount increases. In particular, when a fine mesh-like test pattern as shown in FIG. 5G is used, the amount of calculation is significantly increased. Further, as shown in FIG. 5G, when using a test pattern having a small interval between adjacent line segments, if an angle search is performed with the search range expanded, the adjacent line segments may be erroneously detected.

  Therefore, instead of expanding the search range, an angle search may be performed by adaptively changing the center of the search range. Specifically, the angle search is performed around the result of the adjacent line segment that has been subjected to the angle search immediately before. This is because the change in lens distortion is gradual, and the change in angle is estimated to be negligible between adjacent line segments.

  With reference to FIG. 6 again, the distortion correction processing of the projected image will be described. Next, the projective transformation parameter calculation unit 404 obtains the distance and direction to the projection object based on the coordinates of the four corners calculated as described above, and corrects the trapezoidal distortion of the image projected on the projection object. Projective transformation parameters for calculation are calculated and output to the image correction unit (step S604).

  After that, the image correcting unit 303 performs projective conversion on the image read from the frame memory 302 based on the projective transformation parameter received from the correction amount detecting unit 105, and trapezoidal distortion occurs when the image is projected from the projection unit 101 onto the subject. Corrections are made so as to eliminate them (step S605).

  As described above, according to the projection type image display apparatus 100 according to the present embodiment, the correction amount is based on the projection image of the test pattern photographed by the camera unit 104 arranged at a position different from the irradiation position of the projection unit 101. Can be automatically detected and corrected so that the image irradiated from the projection unit 101 is projected onto the projection object in a correct rectangle.

  Further, according to the projection-type image display apparatus 100 according to the present embodiment, powerful noise reduction processing by autocorrelation is performed on the projection image of the test pattern photographed by the camera unit 104, so that external light such as natural light is emitted. By removing the influence of the background caused by, accurate four corner coordinates can be obtained. Therefore, the projection type image display apparatus 100 can be used not only in a dark room but also in a bright room, and even when projected on a certain wall as well as a pure white screen, an image irradiated from the projection unit 101 Can be corrected so as to be projected onto the projection object in a correct rectangle.

Note that the technology disclosed in the present specification can also be configured as follows.
(1) a projection unit that projects an image onto a projection object;
A camera unit that is installed at a position different from the irradiation position of the projection unit and shoots an image projected on the projection object;
When a test pattern is projected from the projection unit onto the projection target, a background is removed from a test image photographed by the camera unit, and coordinate information relating to the test pattern in the test image after background removal is obtained. A correction amount detection unit that detects and calculates a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
An image correction unit for correcting an image projected from the projection unit based on the correction parameter;
A projection-type image display device comprising:
(2) The correction amount detection unit calculates the coordinates of a plurality of feature points of the test pattern from the test image after background removal, and a projective transformation parameter for correcting distortion included in the projected image of the subject. Is calculated from the coordinates of the plurality of feature points,
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection-type image display device according to (1) above.
(3) The camera unit captures a first test image when a test pattern is not emitted from the projection unit, and displays a second test image when the test pattern is emitted from the projection unit. Shoot,
The correction amount detection unit creates a difference between the first test image and the second test image, removes background information from the second test image, and removes a background including the test pattern. To obtain the test image later,
The projection-type image display device according to (1) above.
(4) The camera unit captures a first test image while irradiating the first test pattern from the projection unit, and irradiates the second test pattern from the projection unit. Sometimes taking a second test image,
The correction amount detection unit creates a difference between the first test image and the second test image, removes background information from the first test image and the second test image, and Obtaining the test image after background removal including a test pattern obtained by synthesizing one test pattern and the second test pattern;
The projection-type image display device according to (1) above.
(5) The correction amount detection unit performs a background information removal process after performing luminance adjustment between the first test image and the second test image.
The projection image display device according to any one of (3) and (4).
(6) The correction amount detection unit performs the luminance adjustment process by multiplying a pixel value of the other image by an average luminance ratio based on one of the first test image and the second test image. Do,
The projection-type image display device according to (5) above.
(7) The correction amount detection unit performs the luminance adjustment processing for each region obtained by dividing the first test image and the second test image in the horizontal and vertical directions, respectively.
The projection type image display device according to (6) above.
(8) The correction amount detection unit normalizes the luminance for a region including a test pattern in the test image after background removal.
The projection-type image display device according to (1) above.
(9) The correction amount detection unit normalizes the luminance only in an area where a variance value of pixel values is equal to or greater than a predetermined threshold in the test image.
The projection-type image display device according to (8) above.
(10) Using a mesh-like test pattern including a plurality of vertical lines and a plurality of horizontal lines,
The correction amount detection unit is configured to calculate a projective transformation parameter for correcting distortion based on the coordinates of a plurality of feature points formed by intersecting points of vertical and horizontal test patterns included in the test image after background removal. Calculate
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection-type image display device according to (5) above.
(11) The test pattern further includes two slits corresponding to a rectangular diagonal line that is an outline of the test pattern.
The projection-type image display device according to (10) above.
(12) The correction amount detection unit estimates a maximum image size that can be projected from the projection unit to the projection object based on the coordinates of the feature points that can be detected from the test image after background removal.
The projection-type image display device according to (10) above.
(13) using a first test pattern including a plurality of vertical lines and a second test pattern including a plurality of horizontal lines;
The correction amount detection unit is configured to calculate a projective transformation parameter for correcting distortion based on the coordinates of a plurality of feature points including intersections of a vertical line and a horizontal line test pattern included in the test image after background removal. Calculate
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection-type image display device according to (4) above.
(14) Each of the first test pattern and the second test pattern includes two slits corresponding to rectangular diagonal lines that form an outline of the test pattern,
The projection-type image display device according to (13) above.
(15) The correction amount detection unit estimates a maximum image size that can be projected from the projection unit to the projection object based on the coordinates of the feature points detected from the test image after background removal.
The projection-type image display device according to (13) above.
(16) The correction amount detection unit further performs noise reduction processing based on autocorrelation on the test image after the background information is removed, and then determines each feature point related to the test pattern in the test image. Detecting coordinate information and calculating correction parameters based on the coordinate information of each feature point;
The projection-type image display device according to (1) above.
(17) The correction amount detection unit performs a noise reduction process by performing an angle search in the vertical direction or the horizontal direction on the test image including a test pattern including a plurality of vertical lines or a plurality of horizontal lines. Calculating the coordinates of a plurality of feature points consisting of each intersection of the test pattern of the line and the horizontal line, and calculating a projective transformation parameter for correcting distortion based on the coordinate information of each intersection;
The projection-type image display device according to (16) above.
(18) The correction amount detection unit detects each test pattern of vertical lines or horizontal lines included in the test image as a quadratic curve.
The projection-type image display device according to (17) above.
(19) The correction amount detection unit performs an angle search centering on a result of an adjacent line segment for which an angle search was performed immediately before.
The projection-type image display device according to (17) above.
(20) projecting a test pattern onto the projection object;
Capturing a test image by capturing a test pattern projected on the projection target; and
Removing a background from the test image;
Detecting coordinate information related to the test pattern included in the test image after removing the background, and calculating correction parameters for correcting the image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
An image projection method comprising:
(21) a procedure for projecting a test pattern onto a projection object;
A procedure for acquiring a test image by photographing a test pattern projected on a projection object,
Removing a background from the test image;
A procedure for detecting coordinate information related to the test pattern included in the test image after removing the background and calculating a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
A computer program written in a computer-readable format that causes a computer to execute.

  As described above, the technology disclosed in this specification has been described in detail with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the scope of the technology disclosed in this specification.

  In the present specification, an embodiment related to a camera-integrated projection-type image display device has been described. However, the technology disclosed in this specification can be applied in the same manner.

  In short, the technology disclosed in the present specification has been described in the form of exemplification, and the description content of the present specification should not be interpreted in a limited manner. In order to determine the gist of the technology disclosed in this specification, the claims should be taken into consideration.

DESCRIPTION OF SYMBOLS 100 ... Projection type image display apparatus 101 ... Projection part 102 ... Image processing part 103 ... Image input part 104 ... Camera part 105 ... Correction amount detection part 201 ... Illumination optical part, 202 ... Liquid crystal panel, 203 ... Projection optical part 301 ... Image Write / read control unit 302 ... Frame memory 303 ... Image correction unit 304 ... Image quality adjustment unit 305 ... Test pattern generation unit 305 ... Output image switching unit 401 ... Shooting image write / read control unit 402 ... Shooting image memory 403 ... feature point calculation unit, 404 ... projection transformation parameter calculation unit

Claims (20)

A projection unit that projects an image onto a projection object;
A camera unit that is installed at a position different from the irradiation position of the projection unit and shoots an image projected on the projection object;
When a test pattern is projected from the projection unit onto the projection target, a background is removed from a test image photographed by the camera unit, and coordinate information relating to the test pattern in the test image after background removal is obtained. A correction amount detection unit that detects and calculates a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
An image correction unit for correcting an image projected from the projection unit based on the correction parameter;
A projection-type image display device comprising: The correction amount detection unit calculates coordinates of a plurality of feature points of the test pattern from the test image after background removal, and sets a plurality of projective transformation parameters for correcting distortion included in the projection image of the subject. Calculated from the coordinates of the feature points of
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection type image display apparatus according to claim 1. The camera unit captures a first test image when not irradiating a test pattern from the projection unit, and captures a second test image when irradiating the test pattern from the projection unit,
The correction amount detection unit creates a difference between the first test image and the second test image, removes background information from the second test image, and removes a background including the test pattern. To obtain the test image later,
The projection type image display apparatus according to claim 1. The camera unit captures a first test image when irradiating the first test pattern from the projection unit, and takes a first test image when irradiating the second test pattern from the projection unit. Take 2 test images,
The correction amount detection unit creates a difference between the first test image and the second test image, removes background information from the first test image and the second test image, and Obtaining the test image after background removal including a test pattern obtained by synthesizing one test pattern and the second test pattern;
The projection type image display apparatus according to claim 1. The correction amount detection unit performs a background information removal process after performing luminance adjustment between the first test image and the second test image.
The projection type image display apparatus according to claim 3. The correction amount detection unit performs the luminance adjustment process by multiplying a pixel value of the other image by a ratio of an average luminance based on one of the first test image and the second test image.
The projection type image display apparatus according to claim 5. The correction amount detection unit performs the luminance adjustment processing for each region obtained by dividing the first test image and the second test image in the horizontal and vertical directions, respectively.
The projection type image display apparatus according to claim 6. The correction amount detection unit normalizes the luminance for an area including a test pattern in the test image after background removal.
The projection type image display apparatus according to claim 1. The correction amount detection unit normalizes the luminance only in a region of the test image in which a variance value of pixel values is a predetermined threshold value or more.
The projection type image display apparatus according to claim 8. Using a mesh-like test pattern containing multiple vertical lines and multiple horizontal lines,
The correction amount detection unit is configured to calculate a projective transformation parameter for correcting distortion based on the coordinates of a plurality of feature points including intersections of a vertical line and a horizontal line test pattern included in the test image after background removal. Calculate
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection type image display apparatus according to claim 3. The test pattern further includes two slits corresponding to a rectangular diagonal line that is an outline of the test pattern.
The projection type image display apparatus according to claim 10. The correction amount detection unit estimates a maximum image size that can be projected from the projection unit onto the projection object based on the coordinates of the feature points detected from the test image after background removal.
The projection type image display apparatus according to claim 10. Using a first test pattern including a plurality of vertical lines and a second test pattern including a plurality of horizontal lines;
The correction amount detection unit is configured to calculate a projective transformation parameter for correcting distortion based on the coordinates of a plurality of feature points including intersections of a vertical line and a horizontal line test pattern included in the test image after background removal. Calculate
The image correction unit performs projective transformation on the image projected from the projection unit using the projective transformation parameter.
The projection type image display apparatus according to claim 4. The correction amount detection unit estimates a maximum image size that can be projected from the projection unit onto the projection object based on the coordinates of the feature points detected from the test image after background removal.
The projection type image display apparatus according to claim 13. The correction amount detection unit further performs noise reduction processing by autocorrelation on the test image after removing background information, and then information on coordinates of each feature point related to the test pattern in the test image And calculating correction parameters based on the coordinate information of each feature point,
The projection type image display apparatus according to claim 1. The correction amount detection unit performs a noise reduction process by performing an angle search in the vertical direction or the horizontal direction on the test image including a test pattern of a plurality of vertical lines or a plurality of horizontal lines, and then performs vertical and horizontal lines. Calculating the coordinates of a plurality of feature points consisting of each intersection of the test pattern, and calculating a projective transformation parameter for correcting distortion based on the coordinate information of each intersection.
The projection type image display apparatus according to claim 15. The correction amount detection unit detects each test pattern of vertical lines or horizontal lines included in the test image as a quadratic curve,
The projection type image display apparatus according to claim 16. The correction amount detection unit performs an angle search around the result of an adjacent line segment that has been subjected to an angle search immediately before,
The projection type image display apparatus according to claim 16. Projecting a test pattern onto the projection object;
Capturing a test image by capturing a test pattern projected on the projection target; and
Removing a background from the test image;
Detecting coordinate information related to the test pattern included in the test image after removing the background, and calculating correction parameters for correcting the image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
An image projection method comprising: A procedure for projecting a test pattern onto a projection object;
A procedure for acquiring a test image by photographing a test pattern projected on a projection object,
Removing a background from the test image;
A procedure for detecting coordinate information related to the test pattern included in the test image after removing the background and calculating a correction parameter for correcting an image projected from the projection unit based on the coordinate information;
Correcting the projected image based on the correction parameters;
A computer program written in a computer-readable format that causes a computer to execute.