WO2006008792A1

WO2006008792A1 - Image projecting method, projector and computer program

Info

Publication number: WO2006008792A1
Application number: PCT/JP2004/010142
Authority: WO
Inventors: Kazuya Arakawa
Original assignee: Sharp Kabushiki Kaisha
Priority date: 2004-07-15
Filing date: 2004-07-15
Publication date: 2006-01-26
Also published as: TW200602858A; TWI344596B

Abstract

An image projecting method for automatically projecting an image while sustaining the aspect ratio when the aspect ratio of an image to be projected is different from that of an object to which the image is to be projected (screen). Four corner positions of a screen S are specified based on an image (3I) picked up at a camera section, an imaginary screen Vs of maximum dimensions is set on the screen S based on that result, the aspect ratio of the screen S and the aspect ratio of an image PJ to be projected with an aspect ratio identical to that of the image to be projected. The size of the image to be projected is then adjusted to coincide with that of the imaginary screen Vs and trapezoidal distortion is corrected.

Description

Specification

Image projection method, projector, and computer program

Technical field

[0001] The present invention automatically adjusts the size of a projected image at the stage of projection preparation.

, An image projection method capable of correcting trapezoidal distortion, and a projector that projects an image by such an image projection method, and for controlling a projector with a general-purpose computer for a control circuit of such a projector Concerning computer programs.

Background art

[0002] In a projector that projects an image onto an object to be projected such as a screen, white wall, and whiteboard, first, a plurality of settings related to projection are prepared as projection preparation so that appropriate projection can be performed from the installation location of the projector. Items need to be adjusted.

[0003] The setting items described above include focus adjustment, color correction, image size adjustment (zoom adjustment), and keystone distortion correction (keystone correction). In setting such items, test pattern images corresponding to the items are sequentially projected from the projector, and the state of the test pattern image projected on the projection object is fed back by imaging with an imaging device. Conventionally, it is generally configured to make adjustments and corrections. For example, in zoom adjustment, the zoom of the projection lens can be adjusted based on the user's instruction or the projector's automatic judgment so that the test pattern image for dimension adjustment projected on the projection object fits on the projection object. Adjust the function to enlarge or reduce the projected image. Note that projectors generally have an optical axis that passes through the lens center of the projection lens that is offset from the center of the projected image. The optical axis passing through the center) is usually used as a reference for adjustment. The projection preparation of such a projector is disclosed in Patent Document 1 below.

Patent Document 1: Japanese Unexamined Patent Publication No. 2000-241874

Disclosure of the invention

Problems to be solved by the invention [0004] By the way, the preparation for projection of the conventional projector as described above is generally performed by recognizing the positions of the four corners of the projection target (for example, the screen) and the four corners of the projected image on the projector side. The size of the projected image and the trapezoidal distortion are adjusted so that the four corners of the projected image coincide with the positions of the four corners of the projection target. By the way, when a TV broadcast image is projected by a projector or an image reproduced from a videotape, DVD, or the like is projected, the aspect ratio of the projected image is generally 4: In addition to 3 in recent years, 16: 9, which is the wide screen of HDTV broadcasting, has become widespread, and even when theatrical movies are broadcast on TV or played back from DVD, the flatter aspect ratio May be adopted. On the other hand, the aspect ratio of an object to be projected, such as a screen, where the range in which the image should be projected is clearly defined is not only the conventional 1: 1 (square), but also the image or video of a TV broadcast. There are 4: 3, 16: 9, etc. based on the assumption that images reproduced from tape, DVD, etc. are projected.

[0005] As described above, in the present situation, the aspect ratio of the projected image and the aspect ratio of the object to be projected, particularly the screen, is actually mixed. The image size was adjusted and the trapezoidal distortion was corrected to match the four corners of the projection object. For this reason, for example, when an image with a aspect ratio of 16: 9 is projected onto a projection object with an aspect ratio of 1: 1, the image is projected in an unnatural state in which the image is extremely stretched vertically. Conversely, when an image with an aspect ratio of 4: 3 is projected onto a projection object with an aspect ratio of 16: 9, the image is projected in an unnatural state stretched in the left-right direction.

[0006] Conventionally, in the above case, it is of course possible to manually adjust the image so that the image is projected in an appropriate range of the projection object with the original aspect ratio. It is necessary to manually adjust the size and correct the trapezoidal distortion when projected onto the projection target, and there is a problem that it is very troublesome.

[0007] The present invention has been made in view of the above problems, and projects a projected image onto a projection object while maintaining the aspect ratio regardless of the aspect ratio of the projection object. The main object of the present invention is to provide an image projection method that can be used and a projector that projects an image using such an image projection method. [0008] Further, in addition to the above-described object, the present invention corrects trapezoidal distortion of a projected image and projects an image onto a virtual projection frame, and projects an image using such an image projection method. The purpose is to provide a projector.

[0009] Further, in addition to the above-described object, the present invention has an object of providing a projector in which a virtual projection frame is set with a maximum dimension. At this time, it is also an object to make it possible to easily set a virtual projection frame by using a known two-dimensional projective transformation.

Another object of the present invention is to provide a computer program for the projector control circuit as described above or for controlling the projector with a general-purpose computer. Means for solving the problem

In order to solve the above problem, an image projection method according to the present invention causes the spatial light modulation means to generate modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and When the modulated light generated by the spatial light modulator is projected onto the rectangular projection object on the projection lens, the modulated light is applied to the spatial light modulator according to information representing an image obtained by deforming the rectangular projection image. In the image projection method of generating and projecting so as to form a rectangular image on the rectangular projection object, the rectangular projection image has the same aspect ratio as the rectangular projection image. A virtual projection frame that is rectangular when projected onto the body is set on the spatial light modulation means.

In order to solve the above-described problem, a projector according to the present invention includes a spatial light modulation unit that generates modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and the spatial A projection lens that projects the modulated light generated by the light modulation unit onto the rectangular projection target, and generates the modulated light in the spatial light modulation unit according to information representing an image obtained by deforming the rectangular projection image. In a projector that projects a rectangular image on the rectangular projection object, the projector has the same aspect ratio as the rectangular projection image, and is projected onto the rectangular projection object. Virtual projection frame setting means for setting a rectangular virtual projection frame on the spatial light modulation means is provided.

In such an image projection method and projector according to the present invention, a virtual projection frame having the same aspect ratio as the projection image is projected even when the aspect ratio of the projection image and the aspect ratio of the projection object are different. Set on the body. [0014] The image projection method according to the present invention causes the spatial light modulation means to generate modulated light in accordance with information representing a rectangular projection image projected onto the rectangular projection object, and the spatial light modulation means When projecting the generated modulated light onto the rectangular projection object onto the projection lens, the spatial light modulation means generates modulated light according to information representing an image obtained by deforming the rectangular projection image, thereby generating the rectangular shape. In the image projecting method for projecting so as to form a rectangular image on the projected object, the projected image has the same external ratio as that of the rectangular projected image, and is projected onto the rectangular projected object. A rectangular virtual projection frame is set on the spatial light modulation means, and is projected with the four corners of the rectangular projection image aligned with the four corners of the virtual projection frame set on the spatial light modulation means. As shown in FIG. The deformation amount of the rectangular projection image represented by the light control is calculated.

[0015] Further, the projector according to the present invention includes a spatial light modulation unit that generates modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and a modulation generated by the spatial light modulation unit. A projection lens that projects light onto the rectangular projection object, and the spatial light modulation means generates modulated light according to information representing an image obtained by deforming the rectangular projection image, thereby generating the rectangular projection object. In a projector that projects a rectangular image on the body, the virtual projection frame has the same aspect ratio as the rectangular projection image and is rectangular when projected onto the rectangular projection object Virtual projection frame setting means for setting the image on the spatial light modulation means, and the four corners of the rectangular projection image coincide with the four corners of the virtual projection frame set on the spatial light modulation means by the virtual projection frame setting means. Projected with As described above, the image processing apparatus includes a calculation unit that calculates a deformation amount of the rectangular projection image represented by the modulated light generated by the spatial light modulation unit.

In such an image projection method and projector according to the present invention, a virtual projection frame having the same aspect ratio as the projection image is projected even when the aspect ratio of the projection image and the aspect ratio of the projection object are different. It is set on the body, and the projected image is projected to the four corners of the set virtual projection frame with the four corners matching.

The image projection method according to the present invention is the image projection method invention described above, wherein the four corner positions of the projection object are imaged by an imaging means, and the four corner positions of the projection object are captured. Specified on the coordinate system set in the image means, and based on the positions of the four corners of the projection object specified on the coordinate system, the aspect ratio of the projection object, and the aspect ratio of the projection image, A virtual projection frame having a maximum size that can be projected onto the projection object with the same aspect ratio as that of the projection image is set.

[0018] Further, in the projector according to the invention described above, the projector according to the present invention is configured such that the imaging unit that images the four corner positions of the projection target and the four corner positions of the projection target are set in the imaging unit. Specifying means on a coordinate system, and the virtual projection frame setting means includes the four corner positions of the projection object specified by the specification means on the coordinate system, the aspect ratio of the projection object, and the Based on the aspect ratio of the projected image, a virtual projection frame having a maximum size that can be projected onto the projection object with the same external ratio as that of the projected image is set.

In such an image projection method and projector according to the present invention, in the above-described image projection method and projector invention, the positions of the four corners of the projection object, the aspect ratio of the projection object, and the projection image specified by the specifying means Based on the aspect ratio, a virtual projection frame of the maximum size that can be set on the projection object is set.

[0020] Further, the image projection method according to the present invention is the image projection method invention described above, wherein the positional relationship of the four corners of the projection object is centered on the center of the projection object and has the same aspect ratio as the projection image. The virtual projection frame is set by converting the positional relationship of the four corners of the rectangle having a position using two-dimensional projective transformation.

Furthermore, the projector according to the present invention is the projector invention described above, wherein the virtual projection frame setting means sets the positional relationship of the four corners of the projection object to the center of the projection object, and the projection image The virtual projection frame is set by converting the positional relationship of the four corners of the rectangle having the same aspect ratio using two-dimensional projective transformation.

In such an image projection method and projector according to the present invention, in the image projection method and projector invention described above, the positional relationship of the four corners of the projection object is centered on the center of the projection object, and the aspect of the projection image A virtual projection frame is set by converting the positional relationship of the four corners of the rectangle having a ratio using a two-dimensional projective transformation.

[0023] Further, the image projection method according to the present invention is the image projection method according to the invention described above. The length of both the upper and lower sides and the length of both left and right sides of the projecting body are obtained on the coordinate system, and the projection target is calculated based on the obtained ratio of the lengths of the upper and lower sides and the left and right sides of the projected body. It is characterized by determining the aspect ratio of the body.

[0024] Further, the projector according to the present invention is the projector invention described above, wherein the lengths of the upper and lower sides and the left and right sides of the projection object are obtained on the coordinate system, and the projection object is based on the ratio between the two. The apparatus further includes means for determining the aspect ratio of the projecting body.

[0025] In such an image projection method and projector according to the present invention, in the image projection method and projector invention described above, the length of both upper and lower sides and the length of both left and right sides of the projection object on the coordinate system are The aspect ratio of the projection object is automatically determined based on the ratio between the two.

[0026] Furthermore, the image projection method according to the present invention is characterized in that, in the above-described image projection method invention, it is determined whether or not the aspect ratio of the projection target is 16: 9.

[0027] Furthermore, the projector according to the present invention is the projector invention described above, wherein the means for determining the aspect ratio of the projection object determines whether or not the aspect ratio of the projection object is 16: 9. It is characterized by doing.

[0028] In such an image projection method and projector according to the present invention, in the above-described image projection method and projector invention, when the aspect ratio of the projection object is automatically determined, it is 16: 9 or not. Only the judgment is made.

[0029] Further, the projector according to the present invention includes a spatial light modulation unit that generates modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and the spatial light modulation unit A projection lens for projecting the modulated light onto the rectangular projection object; and an imaging device, wherein the spatial light modulation means generates modulated light according to information representing an image obtained by deforming the rectangular projection image, and The spatial light modulation based on an image captured by the imaging device with a virtual projection frame having the same aspect ratio as the rectangular projection image for projecting to form a rectangular image on a rectangular projection object In the projector set on the means, the spatial light modulation means generates modulated light representing a test pattern indicating the four corners of the rectangle having the same aspect ratio as the rectangular projection image, and is transmitted from the projection lens to the front. Means for projecting toward a rectangular projection object, and the test pattern is rectangular Means for causing the imaging device to image the state projected toward the projection object, means for detecting the positions of the four corners of the rectangular projection object on the image captured by the imaging device, and the imaging Means for detecting the positions of the four corners of the projected test pattern on the image captured by the apparatus; and the four corners of the rectangular projection object and the four corners of the test pattern on the image captured by the imaging apparatus. And a means for determining four corners of the virtual projection frame set on the spatial light modulation means based on the relative positional relationship.

[0030] In such a projector according to the present invention, test patterns indicating rectangular corners having the same aspect ratio as the projected image are projected toward the projection target, and this state is captured by the imaging device. As a result, the positions of the four corners of the projection object are detected, and the positions of the four corners of the projected test pattern are detected. Then, the four corners of the virtual projection frame are determined based on the relative positional relationship between the four corners of the projection object and the four corners of the test pattern on the image captured by the imaging device.

[0031] Further, the computer program according to the present invention includes a spatial light modulation unit that generates modulated light in accordance with information representing a rectangular projection image projected onto a rectangular projection body, and the spatial light modulation unit generates the spatial light. A projection lens for projecting the modulated light onto the rectangular projection object, and an imaging device, and causing the spatial light modulation means to generate modulated light according to information representing an image obtained by deforming the rectangular projection image. Based on an image captured by the imaging device, a virtual projection frame having the same aspect ratio as the rectangular projection image is projected to a computer that projects a rectangular image on the rectangular projection object. A computer program to be set on the spatial light modulation means, wherein modulated light representing a test pattern indicating the four corners of a rectangle having the same aspect ratio as the rectangular projection image is supplied to the spatial light modulation means. And a procedure for projecting the projection lens from the projection lens toward the rectangular projection object, and a procedure for causing the imaging device to image the state in which the test pattern is projected toward the rectangular projection object. And detecting the positions of the four corners of the rectangular projection object on the image captured by the imaging device, and detecting the positions of the four corners of the projected test pattern on the image captured by the imaging device. The relative positional relationship between the four corners of the procedure and the four corners of the test pattern on the image captured by the imaging device The computer is caused to execute a procedure for determining the four corners of the virtual projection frame set on the spatial light modulator on the basis of the relationship.

In the control of the computer program according to the present invention as described above, test patterns indicating the four corners of a rectangle having the same aspect ratio as the projected image are projected toward the projection object, and this state is captured by the imaging device. Then, the positions of the four corners of the projection object are detected on the image, and the positions of the four corners of the projected test pattern are detected. Then, the computer controls the four corners of the virtual projection frame to be determined based on the relative positional relationship between the four corners of the projection object and the four corners of the test pattern on the image captured by the imaging device.

The invention's effect

[0033] According to the image projection method and the projector according to the present invention as described above, even when the aspect ratio of the projection image and the aspect ratio of the projection object are different, the virtual image having the same aspect ratio as that of the projection image is obtained. Since the projection frame is set on the projection object, it is possible to provide the user with an original image in which the unnaturalness is eliminated compared to the projection state that ignores the conventional external ratio. .

Further, according to the image projection method and the projector according to the present invention, the virtual projection frame having the same aspect ratio as the projection image is projected even when the aspect ratio of the projection image and the aspect ratio of the projection object are different. Since it is set on the body and projected with the four corners of the projected image coincident with the four corners of the set virtual projection frame, unnaturalness has been eliminated compared to the projection state in which the aspect ratio is ignored. The original image can be provided to the user.

[0035] Further, according to the image projection method and projector according to the present invention, in the image projection method and projector invention described above, the result of specifying the positions of the four corners of the projection object on the coordinate system set in the imaging means And the aspect ratio of the projected object and the aspect ratio of the projected image, a virtual projection frame of the maximum size that can be set on the projected object is set. Given the aspect ratio of the image, a virtual projection frame with the maximum size that can be set on the projection object is automatically set, and the image can be projected with the maximum size set in this way. become.

[0036] Further, according to the image projection method and projector according to the present invention, in the above-described image projection method and projector invention, the positional relationship of the four corners of the projection object is determined by a known calculation method. A virtual projection frame is easily set by being transformed using a certain two-dimensional projective transformation.

Furthermore, according to the image projecting method and projector according to the present invention, in the above image projecting method and projector invention, the upper and lower sides of the projection object obtained on the coordinate system set on the image captured by the imaging device Since the aspect ratio of the projection object is automatically determined based on the length of both sides and the length of both sides, the virtual projection frame is automatically set as long as the aspect ratio of the projected image is given. Projection becomes possible.

[0038] Further, according to the image projection method and projector according to the present invention, in the above image projection method and projector invention, when the aspect ratio of the projection object is 16: 9, the vertical length of the projection object is And the vertical length of the projected image should match, otherwise the horizontal length of the projection object and the horizontal length of the projected image should match. Subsequent processing can be performed only by determining whether or not the aspect ratio of the projection object is 16: 9.

[0039] Further, according to the projector of the present invention, the relative positions of the four corners of the projection target on the image captured by the imaging device and the four corners of the rectangular test pattern having the same aspect ratio as the aspect ratio of the projected image. By examining the relationship, the projector automatically performs the process of determining the four corners of the virtual projection frame for projecting the image with the same aspect ratio as the projected image, so the image is stretched horizontally or vertically. It will not be projected in a natural state.

[0040] Further, according to the computer program of the present invention, it is possible to realize the above-described image projection method by controlling the projector as described above or by controlling the projector from the outside with a general-purpose computer. Become.

Brief Description of Drawings

FIG. 1 is a block diagram showing an example of the internal configuration of an embodiment of a projector according to the present invention.

FIG. 2 is a schematic diagram showing a pixel configuration of a spatial light modulation device (panel) included in the projection device unit of the projector according to the present invention.

[Fig. 3] Focus adjustment, zoom adjustment and trapezoidal distortion correction in the projector according to the present invention. It is a schematic diagram of the test pattern image used also for positive (keystone correction).

FIG. 4 is a schematic diagram showing the appearance of a remote control device for a projector according to the present invention.

FIG. 5 is a flowchart showing a processing procedure performed by the projector according to the present invention during normal auto adjustment.

FIG. 6 is a schematic diagram showing a state in which an image is projected onto a screen while maintaining the aspect ratio of the image projected by the projector according to the present invention, taking several combinations of aspect ratios as examples.

FIG. 7 is a schematic diagram showing a state in which a test pattern image is projected onto a screen by the projector according to the present invention.

FIG. 8 is a schematic diagram showing an image when a camera unit captures a state in which a test pattern image is projected onto a screen by a projector according to the present invention.

FIG. 9 is a schematic diagram for explaining the principle of automatically determining the screen aspect ratio in the projector according to the present invention.

FIG. 10 is a schematic diagram showing a state in which a virtual screen is set on the panel coordinate system in the projector according to the present invention.

FIG. 11 is a schematic diagram showing a state in which a virtual screen is set on the panel coordinate system in the projector according to the present invention.

FIG. 12 is a flowchart showing a processing procedure by the system control unit during auto adjustment by the projector according to the present invention.

Explanation of symbols

1 Projector

2 Projection lens

3 Camera

8 Projection device section

8a Spatial light modulation device (panel)

10 System control section

10p program 25 Test pattern image

25b Thick frame (in test pattern image)

S screen

Vs virtual screen

PJ projection image

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the present invention will be described with reference to the drawings showing the best mode for carrying out the invention. FIG. 1 is a block diagram showing an internal configuration example of an embodiment of a projector according to the present invention. The following explanation is an example when the image projection method according to the present invention is implemented by the projector according to the present invention. The image projection method according to the present invention functions not only as a device configured as a projector but also as a projector. For example, the present invention can also be applied to a case where a personal computer is connected to and controlled by a device having both of the above and a projector having only a function of projecting an image.

The projector 1 of the present embodiment has an auto adjustment function that can automatically prepare for projection. Specifically, the auto adjustment function projects a test pattern image from the projection lens 2 onto the projection target screen S during projection preparation, and the camera unit 3 captures the state of the test pattern image projected onto the screen S. This is a function for performing projection preparation such as the size, position, and keystone distortion (keystone) correction of the projected image based on the four corner positions of the screen S obtained as a result and the four corner positions of the test pattern image. The auto adjustment function includes color correction, focus adjustment, and the like. However, since there is no direct relationship with the present invention, description thereof will be omitted.

The projector 1 includes an external connection unit 4 and an image conversion unit 5 as a unit that mainly performs processing on a projection image input from the outside. In addition, the projector 1 includes a color control unit 6, a test pattern image switching unit 7, a projection device unit 8, a projection lens driving unit 9, and a projection lens 2 as parts that mainly perform processes related to projection. . Further, the projector 1 includes a camera unit 3 and a detection unit 11 as a part for performing processing mainly related to the auto adjustment function. Furthermore, the projector 1 receives a user operation. As a stage, an operation unit 12 and a remote control light receiving unit 13 of a remote controller (hereinafter referred to as a remote controller) 20 are provided. The overall control of the projector 1 is performed by the system control unit 10.

The external connection unit 4 is connected to an external device that outputs an image for projection. The external connection unit 4 inputs a rectangular image output from the external device and transmits it to the image conversion unit 5. Based on the control of the system control unit 10, the image conversion unit 5 performs necessary conversion processing such as AZD conversion and transmits the converted image to the projection device unit 8.

[0047] The color control unit 6 performs a process of adjusting the color of the projected image. Specifically, the color control unit 6 adjusts the balance of each color of R (red), G (green), and B (blue) based on the control of the system control unit 10 to thereby adjust the color of the projected image. Make corrections. The test pattern image switching unit 7 generates various test patterns necessary for the auto adjustment function based on the control of the system control unit 10 and transmits the test patterns to the projection device unit 8 as test pattern images.

[0048] The projection device unit 8 incorporates a spatial light modulation device 8a that optically modulates a projection image, that is, information (digital image data) of an image to be projected. The projection device unit 8 generates modulated light obtained by optically modulating digital image data of various images transmitted from the image conversion unit 5, the test pattern image switching unit 7, and the system control unit 10 described later by the spatial light modulation device 8a. To do. The modulated light generated by the spatial light modulation device 8a of the projection device unit 8 in this way is projected onto the external screen S through the projection lens 2. As a result, an image to be projected is displayed on the screen S.

[0049] As the spatial light modulation device 8a, either a liquid crystal panel or a DMD (Digital Micromirrow Device) is generally used. When a liquid crystal panel is used as the spatial light modulation device 8a, each pixel associated with the dot unit of the digital data of the image to be projected transmits the light from the light source while displaying each dot of the image. As a result, modulated light for displaying an image as a whole is projected, and finally an image is displayed on the screen S. When DMD is used as the spatial light modulation device 8a, the light from the light source is reflected while switching the reflection angle of the micromirror (Micromirrow) associated with the dot unit of the digital data of the image to be projected. By projection The image to be formed is projected in a state represented by the entire reflected light (modulated light), and finally the image is displayed on the screen S.

[0050] In the present embodiment, a configuration using a liquid crystal panel as the spatial light modulation device 8a is adopted, and in the following description, an image to be projected is a liquid crystal panel as the spatial light modulation device 8a. The image is projected on the screen S by transmitting the light from the light source to the displayed image and projecting it from the projection lens 2. However, as described above, even when DMD is used, an image is displayed as reflected light (modulated light) as a whole by switching the reflection angle of a micro mirror corresponding to a pixel of digital image data. Therefore, in the same way that individual pixels can be specified in correspondence with the dots of digital image data on the liquid crystal panel, individual micromirrors are specified in correspondence with the dots of digital image data in DMD. It is possible.

FIG. 2 is a schematic diagram showing a pixel configuration of a spatial light modulation device (hereinafter simply referred to as a panel) 8a made of a liquid crystal panel included in the projection device unit 8 described above. As an example in the present embodiment, the panel 8a has a horizontal display range of 1024 pixels in the horizontal direction and 768 pixels in the vertical direction, that is, a rectangular display range conforming to the XGA standard, and the coordinate value (0, 0) of the upper left corner. The panel coordinate system is set with the pixel of) as the origin and the horizontal direction as the X axis and the vertical direction as the y axis. Therefore, when the coordinate values of the panel coordinate system corresponding to each pixel in the horizontal direction and the vertical direction are sent from the system control unit 10 to the projection device unit 8, the projection device unit 8 is based on the coordinate values of the panel coordinate system. Specifies the position and dimensions of the image to be displayed in the display range of panel 8a on the panel coordinate system. For example, if “127” is specified as the horizontal coordinate value and “127” is specified as the vertical coordinate value from the system control unit 10, the projection device unit 8 uses the pixel in the upper left corner of the panel 8a as the origin to A dot is displayed at the position of the 128th pixel in each direction and vertical direction.

[0052] It should be noted that even when a DMD is used as the spatial light modulation device 8a, it is possible to set the same panel coordinate system as when the above-described liquid crystal panel is used. However, as described above, in the present embodiment, since the configuration using the liquid crystal panel is employed as the spatial light modulation device 8a, the configuration in which the liquid crystal panel is used as the spatial light modulation device 8a is also described in the following description. However, the idea about the panel coordinate system is spatial light modulation. The same is true when using a liquid crystal panel as the device 8a and when using a DMD.

[0053] Although not shown, the projection lens 2 is zoomed (image size) in addition to the original lens necessary for enlarging the light beam (modulated light) transmitted through the panel 8a and projecting it on the screen S as an image. It is composed of a plurality of lenses such as an adjustment lens and a focus adjustment lens. The projection lens drive unit 9 has an actuator for changing the positions of the zoom adjustment lens and the focus adjustment lens of the projection lens 2. Then, the projection lens drive unit 9 performs zoom adjustment and focus adjustment by driving the actuator according to the control from the system control unit 10.

Further, the camera unit 3 shown in FIG. 1 captures various test pattern images projected onto the screen S during the automatic adjustment for projection preparation, and transmits the captured images to the detection unit 11. Note that the test pattern image projected from the projector 1 includes zoom adjustment and trapezoidal distortion correction as shown in the schematic diagram of FIG. 3, in addition to the color correction test pattern image and the focus adjustment test pattern (not shown). There is a test pattern image 25 that is also used for (keystone correction). The test pattern image 25 has a thick frame test pattern (hereinafter referred to as a thick frame portion 25b) corresponding to the outline of the projected image, and has various aspect ratios as described later. Are available.

In the following description, the present invention is basically irrelevant to the color correction and focus adjustment processing using the test pattern image for color correction and the test pattern image for focus adjustment. These processes will not be described.

The detection unit 11 analyzes the captured image sent from the camera unit 3. This image analysis is performed on the camera coordinate system. The camera coordinate system is the coordinate system set for camera unit 3. More specifically, the camera coordinate system is a coordinate system set in the imaging field of view of the camera unit 3, and the camera unit 3 is the same as the panel coordinate system set in the spatial light modulation device 8a described above. This is a coordinate system with the upper left corner of the imaging field of view as the origin and the horizontal direction as the X axis and the vertical direction as the y axis. However, the camera coordinate system is actually set with the upper left corner of the image sensor panel (CCD panel) of the camera unit 3 as the origin, which means that the camera coordinates are set on the image captured by the camera unit 3. Can be considered. Therefore, the detection unit 11 uses the conventionally known method based on the image captured by the camera unit 3 to obtain the coordinate values of the four corner positions of the screen S on the camera coordinate system, and the test pattern image 25 in FIG. Coordinate values such as the positions of the four corners of the projected image are detected according to the status of the projector 1 at that time of the thick frame portion 25b. If these coordinate values are detected, the state of trapezoidal distortion of the screen S and the projected image PJ (the thick frame portion 25b of the test pattern image 25) can be obtained by calculation based on the result. Not to mention that. The detection unit 11 transmits the detection result as described above to the system control unit 10.

[0058] The operation unit 12 provided in the projector 1 has a plurality of buttons, switches, and the like. When the user operates these buttons, switches, and the like, the operation unit 12 corresponds to the operated buttons, switches, and the like. Accepts an operation instruction and transmits it to the system control unit 10. Further, the remote control light receiving unit 13 receives an operation signal from the remote control 20 and transmits it to the system control unit 10. FIG. 4 is a schematic diagram showing the external appearance of the remote controller 20. As shown in FIG. 4, the remote control 20 has up / down / left / right selection keys 20a-20d and a decision key 20e in addition to a plurality of buttons, and is displayed on an OSD (On Screen Display) menu image projected from the projector 1. A GUI that allows a user to select a required item from among the displayed items by operating the selection keys 20a to 20d and the decision key 20e is used.

It should be noted that the operation unit 12 is also provided with up / down / left / right selection keys and determination keys similar to those on the remote controller 20. Therefore, when the same operation is performed on the operation unit 12 and the remote controller 20, the same instruction is given to the system control unit 10.

[0060] The system control unit 10 that controls each unit described above includes a ROM10a and a RAM 10b. ROMlOa is used to display a program 10p (computer program according to the present invention) that defines the contents of control performed by the system control unit 10, and various test pattern images and various menu images including the test pattern image 25 shown in FIG. This data is stored in advance. RAMlOb temporarily stores various data generated during control by the system control unit 10.

[0061] The system control unit 10 of the projector 1 of the present embodiment configured as described above performs an adjustment to match the projected image with the screen S by the process shown in the flowchart of FIG. Do. The following processing is stored in ROMlOa. The system control unit 10 executes the program according to the program 10p.

First, the system control unit 10 projects, for example, an all-white image (step Sl).

The detection unit 11 analyzes the image captured by the camera unit 3 in this state, thereby detecting the positions of the four corners of the screen S in the camera coordinate system (step S2). This is easily possible due to the difference between the reflectivity of the screen S and the reflectivity around it (for example, a wall surface).

Next, the system control unit 10 projects a test pattern image 25 for detecting a projected image frame including a thick frame portion 25b that is a test pattern for detecting a projected image frame shown in FIG. 3 (step S3). The detection unit 11 analyzes the image captured by the camera unit 3 in this state, thereby detecting the positions in the camera coordinate system of the four corners of the thick frame portion 25b, which is a test pattern for detecting the projected image frame (step S4). ). This is also possible easily by the difference between the reflectance of the thick frame portion 25b, which is a test pattern for detecting a projected image frame, and the surrounding reflectance. Next, the system control unit 10 performs zoom adjustment based on the positional relationship in the camera coordinate system of each of the four corners of the thick frame 25b, which is a test pattern for detecting the screen S and the projected image frame (step S5). This zoom adjustment is performed, for example, by enlarging or reducing the thick frame portion 25b, which is a test pattern for detecting a projected image frame, by zooming until the thick frame portion 25b, which is a test pattern for detecting a projected image frame, reaches the end point of the screen S. Is done.

[0064] After the zoom adjustment, the detection unit 11 analyzes the positions of the four corners of the thick frame 25b, which is a test pattern for detecting a projected image frame, from the image captured by the camera unit 3 again. (Step S6), the position of the four corners of the screen S detected in step S2 in the camera coordinate system and the position of the four corners of the thick frame 25b after zoom correction detected in step S6. From the relationship, the positions in the panel coordinate system of the four corners of the thick frame portion 25b, which is a test pattern for detecting a projected image frame after zoom correction, that is, coordinate values are determined (step S7). As a result, the image to be projected is displayed on the panel 8a by being deformed in reverse to the shape projected on the screen S.

When the system control unit 10 sequentially executes the above-described series of processes, the projection device unit 8 displays an image on the panel 8a according to the coordinate value given from the system control unit 10, and this is projected Includes a rectangular image with corrected keystone distortion and And projected to the full size of the screen S. Thus, the projection preparation for the projector 1 is automatically completed.

[0066] The projection preparation process by the system control unit 10 as described above is a conventional one, and the screen S does not depend on the aspect ratio of the screen S and the aspect ratio of the projected image (projected image). The size of the projected image and the correction of trapezoidal distortion are adjusted so that the four corners of the image coincide with the four corners. However, if the aspect ratio of the screen S does not match the aspect ratio of the projected image, the image projected on the screen S is not stretched in either the vertical or horizontal direction. It becomes natural.

[0067] Therefore, in the projector 1 according to the present invention, in addition to the conventional general projection preparation process as described above, the aspect ratio of both the screen S and the projected image is considered, and more specifically. It is also possible to project an image with the maximum size on the screen S while maintaining the aspect ratio of the projected image.

[0068] FIG. 6 shows a state in which the projector 1 according to the present invention projects an image on the screen S while maintaining the aspect ratio of an image projected (hereinafter referred to as a projected image PJ). It is the schematic diagram which showed the combination of ratio as an example. Fig. 6 (a) shows the case where the aspect ratio of the screen S is 1: 1 and the aspect ratio of the projected image PJ is 4: 3. In such a case, the projected image PJ is projected so that the length in the left-right direction matches the length in the left-right direction of the screen S, and the length of the screen S in the vertical direction is A part where the image is not projected with 1/8 width. Fig. 6 (b) shows the case where the aspect ratio of the screen S is 1: 1 and the aspect ratio of the projected image PJ is 16: 9. In such a case, the projected image PJ is projected so that the length in the left-right direction matches the length in the left-right direction of the screen S, and the length of the screen S in the vertical direction is There is a part where the image is not projected with the width of 7/32 each. Furthermore, Fig. 6 (c) shows the case where the aspect ratio of the screen S is 4: 3 and the aspect ratio of the projected image PJ is 16: 9. In such a case, the projected image PJ is projected so that the length in the left-right direction matches the length in the left-right direction of the screen S, and the length of the screen S in the up-down direction is applied to the upper and lower ends of the screen S. 3Z2 The part where the image is not projected by 4 width occurs. Furthermore, Fig. 6 (d) shows the screen S. This shows the case where the aspect ratio is 16: 9 and the projected image PJ has an aspect ratio of 4: 3. In such a case, the projected image PJ is projected so that the vertical length of the projected image PJ matches the vertical length of the screen S, and the horizontal length of the screen S is A part where an image is not projected with a width of 2Z16 occurs.

7 and 8 are schematic diagrams for specifically explaining a state in which the thick frame portion 25b, which is a test pattern for detecting a projected image frame, is projected as a test pattern image onto the screen S by the projector 1 according to the present invention. FIG. FIG. 7 is a schematic diagram when the state in which the thick frame portion 25b of the test pattern image 25 is projected onto the screen S is viewed from the front of the screen S. Since the projector 1 is not installed completely opposite to the screen S, the outline of the projection image PJ, in this case, the thick frame portion 25b of the test pattern image 25 has a trapezoidal distortion. Is projected. FIG. 8 is a schematic diagram showing an image captured by the camera unit 3 of the projector 1 in the state shown in FIG. 7, that is, a state seen on the camera coordinate system. On the picked-up image 31, the force S and the screen S are picked up in a state in which the thick frame portion 25b of the test pattern image 25 that is the outline of the projection image PJ is picked up in a substantially rectangular shape.

[0070] By the way, even if the aspect ratio of the projected image PJ is manually set, the aspect ratio of the screen S is determined when the projector 1 is installed in a general situation, for example, indoors. In the installation situation where the screen S is so large and trapezoidal distortion occurs on the image captured by the camera unit 3, it is not normal. For example, when the projector 1 is used in a system called a so-called home theater system or the like, the projector 1 is considered to be installed with the screen S almost facing. Accordingly, it is possible to automatically determine the aspect ratio of the image power screen S imaged by the camera unit 3 as shown in FIG. This will be specifically described below.

FIG. 9 is a schematic diagram for explaining the principle of automatically determining the screen aspect ratio in the projector according to the present invention. As shown in FIG. 9, it is possible by a conventionally known method to specify the coordinate values in the camera coordinate system at the four corners of the screen S based on the captured image 3 I of the camera unit 3. If the coordinate values of the four corners of the screen S are specified, the length of the upper side of the screen S on the camera coordinate system is Hl, the length of the lower side is H2, the length of the left side is VI, and the length of the right side is the same. Can be calculated as V2. Therefore, The aspect ratio of the screen S can be determined by calculating the ratio A (hereinafter referred to as the temporary aspect ratio) between the total length of the top and bottom sides (HI + H2) and the total length of the left and right sides (VI + V2). It is possible to

A = (H1 + H2) / (VI + V2)

[0072] When the actual aspect ratio of the screen S is 1: 1, the above-mentioned temporary aspect ratio A is within a certain range centered on 1/1 (= 1.00), for example, a value between about 0.8-1.2. You can assume that. For example, when the actual aspect ratio of the screen S is 4: 3, the above-mentioned temporary aspect ratio A is in a certain range centered on 4/3 (= 1.33 ...), for example, between about 1.2-1.5. You may assume that it is a value. Further, for example, when the actual aspect ratio of the screen S is 16: 9, the temporary aspect ratio A described above is a certain range centered on 16/9 (= 1.77 ···), for example, about 1.5-1.9. You can assume that the value.

[0073] From the above, the actual aspect ratio of the screen S is determined as follows according to the temporary aspect ratio A. Of course, it is possible to assume values other than 1: 1, 4: 3, and 16: 9 as described above as the actual aspect ratio of the screen S. The standard should be set appropriately according to the same concept as above.

0.8≤A <1.2 → Screen S aspect ratio is 1: 1

1.2≤A <1.5 → Screen S aspect ratio is 4: 3

1.5≤A <1.9 → Screen S aspect ratio is 16: 9

[0074] It should be noted that the vertical length of the screen S and the vertical length of the projected image PJ are made to coincide only when the aspect ratio of the screen S shown in Fig. 6 (d) is 16: 9. When the aspect ratio of the screen S shown in Fig. 6 (a), (b), and (c) is 1: 1 and 4: 3, the horizontal length of the screen S matches the horizontal length of the projected image PJ. I try to let them. Therefore, there is no practical problem even if it is determined only whether the aspect ratio of the screen S is 16: 9.

[0075] Based on the positions of the four corners of the screen S on the camera coordinate system as a result of the analysis of the image taken by the camera unit 3 by the detection unit 11, the system control unit 10 performs the screen S as described above. Determine the aspect ratio. As for the aspect ratio of the image input from the external connection unit 4, some external devices connected to the external connection unit 4 may output a signal for notifying the aspect ratio of the image. When the device is connected to the external connection unit 4, the system control unit 10 can automatically set the image aspect ratio. However, if an external device that cannot notify the projector 1 of the aspect ratio of the image that it outputs is connected to the external connection unit 4, the system control unit 10 uses the default aspect ratio. You may request that the aspect ratio setting be requested in the menu image by using F or OSD.

[0076] In any case, when the aspect ratio of the screen S and the aspect ratio of the projected image are set as described above, the system control unit 10 responds to the set aspect ratio of the image. A test pattern image 25 (actually, the thick frame portion 25b corresponds to the aspect ratio of the image) is generated by the test pattern image switching unit 7 and projected onto the screen S. The result of imaging this state by the camera unit 3 is the state shown in FIGS. However, the above-described processing for determining the actual aspect ratio of the screen S may be performed in a state where the thick frame portion 25b of the test pattern image 25 as shown in FIG. 7 is projected. In this case, the thick frame 25b should be the default aspect ratio or the aspect ratio corresponding to the aspect ratio notified from the external device.

[0077] Based on the camera image 31 as shown in FIG. 8 obtained by capturing the state as shown in FIG. 7 with the camera unit 3, the system control unit 10 maintains the aspect ratio of the projected image PJ on the screen S. The virtual screen Vs to be projected in step S, specifically, the portion corresponding to the projected image PJ in the schematic diagram shown in Fig. 6 (a)-(d) is set by applying a known two-dimensional projective transformation. Then, a projection image PJ in which the trapezoidal distortion is corrected so as to coincide with the dimensions of the virtual screen Vs is projected. In FIG. 8, the aspect ratio of the screen S is 4: 3, and the aspect ratio of the image (projected image PJ) displayed on the screen S (same as the aspect ratio of the virtual screen Vs) is 16: 9. Explain the case.

Specifically, a known two-dimensional projective transformation is applied to the positions of the four corners of the screen S on the image captured by the camera unit 3 (hereinafter referred to as coordinate values on the force coordinate system). Thus, the positions of the four corners of the virtual screen Vs (the coordinate values of the camera coordinate system) are obtained, and the obtained virtual Projection after correcting trapezoidal distortion to the four corner positions of the screen Vs (camera coordinate system coordinate values) The known two-dimensional projections so that the four corner positions of the PJ (camera coordinate system coordinate values) match. Perform shadow conversion.

First, a calculation method for determining the coordinate values of the four corners of the virtual screen Vs will be described. Needless to say, this calculation itself is executed by the system control unit 10 based on the result of the detection unit 11 analyzing the image captured by the camera unit 3. The input parameters are the coordinate values of the four corners of the screen S on the camera coordinate system, the coordinate values of the four corners of the projected image PJ, and the X direction resolution and y direction solution of the panel 8a of the projection device unit 8 of the projector 1. Each image is defined as follows (see Fig. 8).

[0080] · Coordinate values of the four corners of screen S on the camera coordinate system:

(sxl, syl) sx2, sy2) 8 sx3, sy3), (sx4, sy4)

• Coordinate values of the four corners of the projected image PJ on the camera coordinate system:

(pjxl, pjyl), (pjx2, pjy2), (pjx3, pjy3), (pjx4, pjy4)

'Panel x resolution: col

• y resolution of the panel: row

Note that the values of row and col corresponding to the aspect ratio of the screen S are as follows.

Screen aspect it: row: col

1: 1 800 800

4: 3 768 1024

16: 9 900 1600

By the way, as shown in FIG. 2, the coordinate values (coordinate values on the panel coordinate system) of the four corners (PI, P2, P3, P4) of the panel 8a of the projection device unit 8 are as follows. In the embodiment, it is possible to express with the resolution conforming to the XGA standard), and the respective coordinate values (coordinate values of the screen S on the camera coordinate system) are (Pxl, Pyl), (Px2, Py2 ), (Px3, Py3), (Px4, Py4). Note that PX is defined as a coordinate value at an arbitrary position on the panel 8a (a coordinate value on the panel coordinate system), and a conversion destination thereof, that is, a coordinate value on the camera coordinate system is defined as (LXX, LYY).

[0083] Panel at the four corners of the panel → destination coordinates

Coordinate value in coordinate system (coordinate value of screen on camera coordinate system) PI = (0, 0) → (Pxl, Pyl)

P2 = (col, 0) → (Px2, Py2)

P3 = (col, row) → (Px3, Py3)

P4 = (0, row) → (Px4, Py4)

PX = (LX, Ly) → (LXX, LYY)

However, since the conversion destination coordinate values of the coordinate values in the panel coordinate system at the four corners of the panel 8a described above are the coordinate values of the screen S on the camera coordinate system, the following relationship is established.

Px 丄 = sxl, Pyl = syl

Px2 = sx2, Py2 = sy2

Px3 = sx3, Py3 = sy3

Px4 = sx4, Py4 = sy4

[0085] Next, coordinate values normalized for use in the calculation (coordinate values for calculation), that is, the coordinates of the four corners of the panel 8a (panel coordinate system) are 0 and 1 for both the X and y axes. The value obtained by converting to a value between and normalized is obtained, and the above-mentioned conversion destination coordinate value (the coordinate value in the camera coordinate system) is also normalized. The coordinate values of the four corner positions pl, p2, p3, p4 of the panel represented by the coordinate values normalized on the panel coordinate system and the normalized position px on the panel are the original coordinate values (panel coordinate values ) And the coordinate value obtained by normalizing the original coordinate value and the conversion destination coordinate value (the coordinate value of the camera coordinate system) are expressed as follows.

[0086] Original coordinate value: Conversion coordinate value

pl: (0, 0): (xl, yl) where (xl, yl) = (0, 0)

_P 2: (l, 0): (x2, y2)

p3: (l, l): (x3, y3)

p4: (0, 1): (x4, y4)

px: (χ, y): (χχ, yy)

[0087] The above relation is such that the calculation is simplified by offsetting one point (xl, yl) of the conversion destination to the corresponding one point (0, 0) in the original coordinate system. This is a general method for two-dimensional projective transformation. The normalized destination coordinate values x, y xl, χ2, χ3, χ4 and yl, y2, y3, y4 are the coordinate values of the four corners of the screen S on the camera coordinate system described above ( Pxl, Pyl), (Px2, Py2), (Px3, Py3), (Px4, Py4) and the row and col values according to the aspect ratio of the screen S are expressed as follows.

[0088] xl = 0 where xl = (Pxl-Pxl) / col

x2 = (Px2-Pxl) ol

x3 = (Px3-Pxl) ol

x4 = (Px4-Pxl) ol

yl = 0 where yl = (Pyl-Pyl) / row

y2 = (Py2-Pyl) / row

y3 = (Py2-Pyl) / row

y4 = (Py4-Pyl) / row

[0089] From the above relationship, conversion coefficients a, b, c for converting the original coordinate value (the coordinate value in the panel coordinate system) into the conversion destination coordinate value (the coordinate value of the screen S on the camera coordinate system). , al, a2, bl, b2, aO, b0, cO are obtained as follows. However, here, since it is a conversion coefficient for converting the original coordinate value (panel coordinate value) to the conversion destination coordinate value, a positive conversion coefficient is obtained. Note that a, b, and c are intermediate constants that are used to reduce redundant calculations even in the known two-dimensional projective transformation.

a = (x3 * y4-x4 * y3): Intermediate constant

b = (x2 * y3-x3 * y2): Intermediate constant

c = (x2 * y4-x4 * y2): Intermediate constant

al = a * x2

a2 = a * y2

bl = b * x4

b2 = b * y4

aO =-b + c

bO = ― a + c

cO = b + a-c

Next, the position on the panel 8a of the projection device unit 8 corresponding to the positions of the four corners of the virtual screen Vs, that is, the coordinate value in the panel coordinate system is obtained. Note that the virtual script on panel 8a Define the coordinate value of the position corresponding to the four corners of the Vs as (new xi, new yi) and its positive transformation coordinate value (coordinate value in the camera coordinate system) as (virtual xi, virtual yi). . Here, i is 1, 2, 3, 4 and the 100% coordinate value is the coordinate value of the position on the panel 8a corresponding to the position of the four corners of the screen S, specifically, the coordinate of the four corners of the panel 8a. Value.

[0092] 100% coordinate value → virtual screen coordinate value → forward conversion coordinate value

(On panel) (Panel coordinate system) (Camera coordinate system)

P1: (0, 0) → new xl, new yl) → (virtual xl, virtual yl)

P2: (col, 0) → (new x2, new y2) → (virtual x2, virtual y2)

P3: (col, row) → (new x3, new y3) → virtual x3, virtual y3) P4: (0, row) → (new x4, new y4) → virtual x4, virtual y4)

From the above results, the panel coordinate values (new xi, new yi) of the virtual screen Vs are obtained according to the combination of the aspect ratio of the screen S and the aspect ratio of the projected image. In this case, the actual resolution of panel 8a is 1024 x 768, which is assumed to be 800 x 800 if the screen S aspect ratio is 1: 1, and 1 if the screen S aspect ratio is 4: 3 Assuming 024 x 768, the aspect ratio of screen S: 9 If 1600 x 900, assume the panel coordinate value of virtual screen Vs. 10 and 11 are schematic diagrams showing a state in which the virtual screen Vs is set on the panel coordinate system.

[0094] (1) When the aspect ratio of screen S is 1: 1 and the aspect ratio of projected image PJ is 4: 3. In this case, row = 800 and col = 800 are assumed. See Figure 10 (a). However, the value shown in parentheses in Fig. 10 (a) is the value when the above assumption is made, not the actual coordinate value of panel 8a.

new xl = 0, new yl = 100

new x2 = 799, new y2 = 100

new x3 = 799, new y3 = 699

new x4 = 0, new y4 = 699

[0095] (2) When the aspect ratio of screen S is 1: 1 and the aspect ratio of projected image PJ is 16: 9. In this case, assume row = 800, col = 800. See Figure 10 (b). However, the value indicated by () in Fig. 10 (b) is not the actual coordinate value of panel 8a, but the above assumption is made. Value.

new xl = 0, new yl = 175

new x2 = 799, new y2 = 175

new x3 = 799, new y3 = 624

new x4 = 0, new y4 = 624

[0096] (3) When the aspect ratio of screen S is 4: 3 and the aspect ratio of projected image PJ is 16: 9. In this case, assume row = 768, col = 1024. See Figure 11 (a). However, the values shown in parentheses in Fig. 11 (a) are straightforward when the above assumptions are made, rather than the actual coordinate values of panel 8a.

new xl = 0, new yl = 96

new x2 = 1023, new y2 = 96

new x3 = 1023, new y3 = 671

new x4 = 0, new y4 = 671

[0097] (4) When the aspect ratio of the screen S is 16: 9 and the aspect ratio of the projected image PJ is 4: 3. In this case, assume row = 900, col = 1600. See Figure 11 (b). However, the values shown in parentheses in Fig. 11 (b) are straightforward when the above assumptions are made, rather than the actual coordinate values of panel 8a.

new xl = 200, new yl = 0

new x2 = 1399, new y2 = 0

new x3 = 1399, new y3 = 899

new x4 = 200, new y4 = 899

Next, the coordinate values of the four corners of the virtual screen Vs on the camera image 31 captured by the camera unit 3, that is, the coordinate values of the four corners of the virtual screen Vs in the camera coordinate system are obtained.

[0099] Specifically, the virtual screen Vs on the camera image, that is, the camera coordinate system, is obtained by performing positive conversion on the coordinate values of the four corners of the screen S in the panel coordinate system on the panel 8a. Find the coordinate values of the four corners. However, the coordinate values LX and LY of the arbitrary point PX in the panel coordinates (original coordinates) are the coordinate values new xl one new x4, new yl one new y4 of the virtual screen Vs, respectively. Arbitrary point PX conversion coordinates (camera seat) The coordinate values LXX and LYY of the standard system) are converted to positive values, and the virtual screen Vs corners VP1, VP2, VP3, and VP4 in the camera coordinate system are coordinate values virtual xl one virtual x4, virtual yl one virtual y4, and finally Specifically, the coordinate values LXX and LYY of the four corners VP1, VP2, VP3, and VP4 of the virtual screen Vs in the camera coordinate system are obtained as follows.

[0100] LX: new x 丄, new x2, new χό, new x4

LY: new yl, new y2, new y3, new y4

LXX: virtual xl, virtual x2, virtual x3, virtual x4

LYY: virtual yl, virtual y2, virtual y3, virtual y4

[0101] x = LX ol where LX = 0 '' col

y = LY / row where LY = 0 "-row

z = aO * x + b0 * y + cO

xx = (al * x + bl * y) / z

yy = (a2 * x + b2 * y) / z

Therefore,

LXX = col * xx + Pxl

LYY = row * yy + Pyl

That is, by setting LX to new xi, virtual xi can be found as LXX, and by setting LY as new yi, virtual can be found as LYY.

[0103] As described above, the coordinate values of the four corners of the virtual screen Vs in the camera coordinate system (virtual xl

(virtual x4, virtual yl one virtual y4) is obtained, so that the two-dimensional By performing the projective transformation, the projected image PJ is corrected for trapezoidal distortion and projected in accordance with the virtual screen Vs with the same size as the virtual screen Vs. This two-dimensional projective transformation itself is a known technique.

Specifically, the projection image PJ can be projected in accordance with the virtual screen Vs by performing the following calculation. Needless to say, this calculation itself is executed by the system control unit 10 based on the result of the detection unit 11 analyzing the image captured by the camera unit 3. [0105] The input parameters are the coordinate values of the four corners of the virtual screen Vs on the camera coordinate system, the coordinate values of the four corners of the projection image PJ, and the X-direction and y-direction resolutions of the panel 8a. Each is defined as follows. However, in the following calculations, the resolutions col and row in the X direction and y direction of the panel 8a differ from the actual panel 8a values depending on the combination of the aspect ratio of the projected image PJ and the panel resolution. Use such values.

• When the actual panel resolution is XGA (1024 X 768)

When the aspect ratio of the projected image PJ is 4: 3, col = 1024, row = 768.

When the aspect ratio of the projected image PJ is 16: 9, col = 1024, row = 576.

-When the actual panel resolution is SVGA (800 X 600)

When the aspect ratio of the projected image PJ is 4: 3, col = 800, row = 600.

When the aspect ratio of the projected image PJ is 16: 9, col = 800, row = 450.

[0106] · Coordinates of the four corners of the virtual screen Vs on the camera coordinate system:

(virtual xl, virtual yl), (virtual x2, virtual y2),

(virtual x3, virtual y3), (virtual x4, virtual y4)

'Coordinates of the four corners of the projected image PJ (bold frame 25b) on the camera coordinate system:

(pjxl, pjyl), (pjx2, pjy2), (pjx3, pjy3), (pjx4, pjy4)

[0107] Here, the coordinate values of the four corners (Pl, P2, P3, P4) of the virtual screen Vs on the camera coordinate system

virtual xl, virtual yl), ^ virtual x2, virtual y2), ^ virtual χό, virtual y3), ^ virtual x4, virtual y4) to (Pxl, Pyl), (Px2, Py2), (Px3, Py3), ( Px4, Py4), and the conversion destination coordinate values (coordinate values on the panel coordinate system) are (0, 0), (col, 0), (col, row), (0, row). Note that PX is a coordinate value at an arbitrary position on the camera coordinate system.

The coordinate value of the virtual screen on the camera coordinate system

→ Conversion destination coordinate value (coordinate value on the panel coordinate system)

VP1 = (Pxl, Pyl) → (0, 0)

VP2 = (Px2, Py2) → (col, 0)

VP3 = (Px3, Py3) → (col, row)

VP4 = (Px4, Py4) → (0, row)

PX = (LXX, LYY) → (LX, LY) However, since the coordinate values of the four corners (VP1, VP2, VP3, VP4) of the virtual screen Vs on the camera coordinate system should match the coordinates of the four corners of the projected image PJ on the camera coordinate system, The following relationship holds.

P l = pj l, Pyl = pjyl

Px2 = pjx2, Py2 = pjy2

Px3 = pjx3, Py3 = pjy3

Px4 = pjx4, Py4 = pjy4

[0109] Next, coordinate values normalized for use in the calculation (coordinate values for calculation), that is, the coordinate values of the four corners of the virtual screen Vs on the camera coordinate system are 0 and 1 for both the x and y axes. The normalized value is obtained by converting to a value between and the panel coordinate value, which is the conversion destination coordinate value described above, is normalized in the same manner. The coordinates of the four corners pl, p2, p3, p4 of the virtual screen Vs on the camera coordinate system represented by the normalized coordinate values in this way and any normalized point px on the camera coordinate system The standard coordinates (camera coordinate system) and the coordinate values obtained by normalizing the coordinate values of the original coordinates and the conversion destination coordinates (coordinate values of the panel coordinate system) are expressed as follows.

[0110] Original coordinate value: Conversion destination coordinate value

pl: (0, 0): (xl, yl) where (xl, yl) = (0, 0)

_P 2: (l, 0): (x2, y2)

_P 3: (l, l): (x3, y3)

_P 4: (0, 1): (x4, y4)

px: (χ, y): (χχ, yy)

[0111] The above relationship simplifies the calculation by offsetting one point (xl, yl) of the conversion destination to the corresponding point (0, 0) on the original coordinate system. This is a general technique in the known two-dimensional projective transformation. The normalized destination coordinate values x and y xl, x2, x3, x4 and yl, y2, y3, y4 are the coordinate values of the four corners of the virtual screen Vs on the camera coordinate system (Pxl, Pyl ), (Ρχ2, Py2), (Px3, Py3), and (Px4, Py4).

[0112] xl = 0 However, xl = (Pxl—Pxl) ol

x2 = (Px2-Pxl) ol x3 = (Px3-PxD / col

x4 = (Px4-PxD / col

yl = 0 where yl = (Pyl-Pyl) / row

y2 = (Py2-Pyl) / row

y3 = (Py2-Pyl) / row

y4 = (Py4-Pyl) / row

[0113] From the above relationship, conversion coefficients a, b, c, al, for converting the original coordinate value (camera coordinate system) to the conversion destination coordinate value (the coordinate value of the virtual screen Vs on the panel coordinate system) a2, bl, b2, a0, b0, cO are obtained as follows. However, here, since it is a conversion coefficient for converting the original coordinate value (virtual screen Vs coordinate value on the camera coordinate system) to the conversion destination coordinate value (panel coordinate system), a positive conversion coefficient is obtained. At the same time, inverse transform coefficients n aO, n b0, n c0, n al, n b l, n a2, n b2 are also obtained. Note that a, b, and c are intermediate constants used to reduce duplicate calculations even in the known two-dimensional projective transformation.

[0114] · Positive conversion coefficient

a =: (x3 * y4--x4 * y3): Intermediate constant

b =: (x2 * y3-x3 * y2): Intermediate constant

c =: (x2 * y4--x4 * y2): Intermediate constant

al = a * x2

a2 = a * y2

bl = b * x4

b2 = b * y4

aO =-b + c

bO =-a + c

cO = b + a ― c

[0115] · Inverse transformation coefficient

n aO = (aO * b2 _ a2 * b2)

n bO = (al * bO _ aO * b l)

n cO = (a2 * bl-al * b2) Next, the corresponding coordinate values on the panel coordinate system are calculated by inversely transforming the coordinate values of the four corners of the virtual screen Vs on the camera coordinate system. However, the coordinate values LXX and LYY of an arbitrary point PX on the camera coordinate system (original coordinates) are set as the coordinate values sxi and syi of the four corners of the virtual screen Vs, respectively. The coordinate values LX and LY of the four corners on the panel coordinate system corresponding to the four corners of the virtual screen Vs are obtained as follows.

[0117] XX = (LXX-Pxl) ol where LXX = 0---col

yy = (LYY-Pyl) / row where LYY = 0---row

zz = n aO * xx + n D0 * yy + n cO

x = (n al * xx + n bl * yy) / zz

y = (n a2 * xx + n b2 * yy) / zz

Therefore,

LX = col * x

LY = row * y

[0118] As described above, it is possible to project the projected image PJ so as to coincide with the virtual screen Vs.

FIG. 12 is a flowchart showing a processing procedure by the system control unit at the time of auto adjustment by the projector according to the present invention. Specifically, FIG. 12 is a flowchart showing the processing procedure by the system control unit 10 including the processing of projecting onto the screen S while maintaining the aspect ratio of the projected image PJ as described above. The control shown in this flowchart is processed according to the program 10p stored in ROMlOa.

[0120] In the present embodiment, the system control unit 10 is configured to process each procedure according to the program 10p stored in the ROMlOa. However, the projector of the present invention includes all of these procedures. Of course, it is possible to adopt a configuration in which part or all of the data is processed by dedicated hardware (dedicated circuit).

[0121] First, the user places the projector 1 in front of the screen S, and operates the operation unit 12 or the remote control 20 to give an instruction to perform automatic adjustment for projection preparation to the projector. At this time, the projection adjustment is automatically adjusted to match the size of the screen S with the projection image PJ One of two types can be selected: normal auto adjustment and auto adjustment that projects onto the screen S while maintaining the aspect ratio of the projected image PJ as described above.

[0122] The system control unit 10 monitors whether or not an instruction to perform automatic adjustment for projection preparation while maintaining the aspect ratio of the image and other instructions have been received (step S11). When an instruction other than an instruction to perform automatic adjustment for projection preparation while maintaining the image aspect ratio is accepted (NO in step S11), the system control unit 10 executes processing corresponding to the accepted instruction (step S12). . This includes, for example, normal projection preparation for matching the size of the projected image PJ with the size of the screen S. On the other hand, when receiving an instruction to perform automatic adjustment for projection preparation while maintaining the external ratio of the projected image PJ (YES in step S11), the system control unit 10 performs color correction and focus adjustment items. In addition to the automatic adjustment, automatic adjustment for projection preparation while maintaining the aspect ratio of the projection image PJ described above is started (step S13). In the following description, descriptions regarding color correction and focus adjustment are omitted.

[0123] At the start of auto adjustment, the system control unit 10 first detects the positions (coordinate values) in the camera coordinate system of the four corners of the screen S from the image captured by the camera unit 3 (step S14), and then The aspect ratio of the screen S is judged (step S15). Subsequently, the system control unit 10 detects the positions (coordinate values) in the camera coordinate system of the four corners of the thick frame portion 25b, which is a test pattern for detecting a projected image frame (step S16). The process of detecting the positions (coordinate values) of the four corners of the screen S and the projected thick frame 25b by the system control unit 10 in the camera coordinate system is the same as in the flowchart of FIG. The aspect ratio of the screen S is determined by the method described above based on the coordinate values of the four corners of the screen S in the camera coordinate system detected by the detection unit 11.

[0124] Next, the system control unit 10 determines whether or not the screen size of the projection image is set (step S17). Here, the screen size of the projected image is a size determined in accordance with a display mode for designating what aspect ratio the image to be projected is displayed on the screen. For example, when the zoom mode is specified, the screen size of the projected image with an aspect ratio of 4: 3 remains the same as 4: 3 but is enlarged as a whole. For example, when full mode is specified, the projected image with an aspect ratio of 16: 9 is horizontal ( The image is magnified by about 1.3 times in the horizontal direction, and only the central part in the horizontal direction is displayed on the screen S with an aspect ratio of 4: 3. Furthermore, for example, when the squeeze (full) mode is designated, the squeeze-recorded image is displayed on the screen S with an aspect ratio of 16: 9. In this way, the image that should be projected can be displayed in various modes according to the user's preference and displayed on the screen S. And the mode (screen size) set by the user determine the aspect ratio of the image to be displayed on the screen S.

[0125] If the screen size is set (YES in step S17), system control unit 10 advances the process to step S18 described later. However, if the screen size is not set (NO in step S17), the system control unit 10 requests the user to set the screen size of the projection image (step S21). As an example of a specific method, the system control unit 10 requests input of the aspect ratio of the projected image from the OSD menu image, and enters a standby state (NO in step S22). When the user operates the operation unit 12 or the remote controller 20 to input the screen size of the projection image PJ (YES in step S22), the system control unit 10 sets the input screen size as the screen size of the projection image PJ.

[0126] As a result, the aspect ratio of the projected image PJ is determined from the relationship between the set screen size and the aspect ratio of the image to be projected, so that the system control unit 10 uses the virtual screen Vs screen as described above. Determine the coordinate values of the four corners on S (Step S18), and adjust the zoom so that the determined virtual screen Vs matches the projected image PJ, specifically, the thick frame 25b around the test pattern image 25. Then, trapezoidal distortion correction is performed (step S19). Thereafter, various images input from the external connection unit 4 can be projected onto the virtual screen Vs on the screen S according to the user's instructions (step S20).

[0127] In the above-described embodiment, the virtual screen Vs (virtual projection frame) is set based on the image captured by the camera unit 3, but for the screen S (projected body), for example, the screen S Photodetectors such as photodiodes are installed at the four corners to detect the positions of the four corners of the screen S, and zoom operations and lens shifts are performed on the projected image (specifically, the thick frame 25b that is the test pattern). By configuring so that the four corners are detected by an operation or the like, the present invention can be applied. In the above-described embodiment, the four corners of the projected image (specifically, the thick frame portion 25b, which is a test pattern) are aligned with the four corners of the virtual screen Vs (virtual projection frame), thereby reducing the trapezoidal distortion of the projected image. Of course, the virtual screen Vs can be used for zoom adjustment. In this case, zoom adjustment can be performed in a state where the center of the projected image coincides with the center of the screen S (projected body).

Claims

The scope of the claims

[1] Spatial light modulation means generates modulated light in accordance with information representing a rectangular projection image projected onto a rectangular projection object, and the modulated light generated by the spatial light modulation means is used as the rectangular object. When projecting the projection lens onto the projection lens, the spatial light modulation means generates modulated light according to information representing an image obtained by deforming the rectangular projection image, and the rectangular image is formed on the rectangular projection object. In the image projection method of projecting so that

A virtual projection frame having the same aspect ratio as the rectangular projection image and having a rectangular shape when projected onto the rectangular projection object is set on the spatial light modulation means. Image projection method.

[2] According to information representing a rectangular projection image projected onto the rectangular projection object, the spatial light modulation unit generates modulated light, and the modulated light generated by the spatial light modulation unit is used as the rectangular target object. When projecting the projection lens onto the projection lens, the spatial light modulation means generates modulated light according to information representing an image obtained by deforming the rectangular projection image, and the rectangular image is formed on the rectangular projection object. In the image projection method of projecting so that

A virtual projection frame having the same aspect ratio as the rectangular projection image and having a rectangular shape when projected onto the rectangular projection object is set on the spatial light modulation unit, and the spatial light modulation unit The rectangular projection image represented by the modulated light generated by the spatial light modulation means is projected so that the four corners of the rectangular projection image coincide with the four corners of the virtual projection frame set above. Calculate the amount of deformation

An image projection method characterized by the above.

[3] The four corner positions of the projection object are imaged by an imaging means,

The positions of the four corners of the projection object are specified on the coordinate system set in the imaging means, the four corner positions of the projection object specified on the coordinate system, and the aspect ratio of the projection object The virtual projection frame having the maximum size that can be projected onto the projection object with the same aspect ratio as that of the projection image is set based on the aspect ratio of the projection image. Image projection method.

[4] The two-dimensional projective transformation is used for the positional relationship of the four corners of the projection object with respect to the positional relationship of the four corners of the rectangle having the same aspect ratio as the projection image with the center of the projection object as the center. The image projection method according to claim 3, wherein the virtual projection frame is set by performing conversion.

[5] The length of both the upper and lower sides and the length of both left and right sides of the projection object are obtained on the coordinate system, and based on the obtained ratio of the lengths of the upper and lower sides and left and right sides of the projection object. To determine the external ratio of the projection object

The image projection method according to claim 3, wherein:

6. The image projecting method according to claim 5, wherein it is determined whether or not the aspect ratio of the projection object is 16: 9.

[7] Spatial light modulation means for generating modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and the modulated light generated by the spatial light modulation means for the rectangular projection A projection lens that projects onto the body, and causes the spatial light modulation means to generate modulated light according to information representing an image obtained by deforming the rectangular projection image, thereby generating a rectangular image on the rectangular projection object. In the projector that projects so that

Virtual projection frame setting means for setting, on the spatial light modulation means, a virtual projection frame having the same aspect ratio as the rectangular projection image and having a rectangular shape when projected onto the rectangular projection object A projector comprising:

[8] Spatial light modulation means for generating modulated light according to information representing a rectangular projection image projected onto a rectangular projection object, and the modulated light generated by the spatial light modulation means for the rectangular projection A projection lens that projects onto the body, and causes the spatial light modulation means to generate modulated light according to information representing an image obtained by deforming the rectangular projection image, thereby generating a rectangular image on the rectangular projection object. In the projector that projects so that

Virtual projection frame setting means for setting, on the spatial light modulation means, a virtual projection frame having the same aspect ratio as the rectangular projection image and having a rectangular shape when projected onto the rectangular projection object When,

The modulation generated by the spatial light modulation means so that the four corners of the rectangular projection image coincide with the four corners of the virtual projection frame set on the spatial light modulation means by the virtual projection frame setting means. Calculation means for calculating the deformation amount of the rectangular projection image represented by light; A projector comprising:

[9] Imaging means for imaging the positions of the four corners of the projection object;

Specifying means for specifying the positions of the four corners of the projection object on a coordinate system set in the imaging means;

The virtual projection frame setting means is based on the four corner positions of the projection object specified by the specification means on the coordinate system, the aspect ratio of the projection object, and the aspect ratio of the projection image. Set the maximum virtual projection frame that can be projected onto the projection object with the same aspect ratio as

The projector according to claim 7 or 8, characterized in that.

[10] The virtual projection frame setting means sets the positional relationship of the four corners of the projection object to the positional relationship of the rectangular four corners having the same aspect ratio as the projection image with the center of the projection target as the center. 10. The projector according to claim 9, wherein the virtual projection frame is set by performing transformation using a three-dimensional projective transformation.

[11] A means for determining the length of both upper and lower sides and the length of both right and left sides of the projection body on the coordinate system, and further determining means for determining the external ratio of the projection body based on a ratio between the two. The projector according to claim 9, comprising the projector.

12. The projector according to claim 11, wherein the means for determining the aspect ratio of the projection object determines whether or not the aspect ratio of the projection object is 16: 9.

[13] Spatial light modulation means for generating modulated light in accordance with information representing a rectangular projection image projected onto a rectangular projection object, and the modulated light generated by the spatial light modulation means for the rectangular projection A projection lens that projects onto the body, and an imaging device, wherein the spatial light modulation means generates modulated light according to information representing an image obtained by transforming the rectangular projection image on the rectangular projection object. In a projector that sets a virtual projection frame having the same aspect ratio as that of the rectangular projection image on the spatial light modulator on the basis of an image captured by the imaging device for projecting to form a rectangular image. ,

Means for causing the spatial light modulation means to generate modulated light representing test patterns indicating the four corners of a rectangle having the same aspect ratio as the rectangular projection image, and projecting the modulated light from the projection lens toward the rectangular projection object When, Means for causing the imaging device to image the state in which the test pattern is projected toward the rectangular projection object;

Means for detecting positions of four corners of the rectangular projection object on an image captured by the imaging device;

Means for detecting positions of four corners of the projected test pattern on an image captured by the imaging device;

The virtual projection frame set on the spatial light modulation means based on the relative positional relationship between the four corners of the rectangular projection object and the four corners of the test pattern on the image captured by the imaging device Means to determine the four corners of

A projector comprising:

Spatial light modulation means for generating modulated light according to information representing a rectangular projection image projected onto the rectangular projection object, and projecting the modulated light generated by the spatial light modulation means onto the rectangular projection object A projection lens and an imaging device, wherein the spatial light modulation means generates modulated light according to information representing an image obtained by transforming the rectangular projection image, and the rectangular projection on the rectangular projection object is generated. A computer program that causes a computer to project an image so that a virtual projection frame having the same external ratio as the rectangular projection image is set on the spatial light modulation unit based on an image captured by the imaging device. The spatial projection means generates modulated light representing a test pattern indicating four corners of a rectangle having the same outer ratio as the rectangular projection image, and the rectangular projection object is generated from the projection lens. Heading A procedure for the projection Te,

A procedure for causing the imaging device to image a state in which the test pattern is projected toward the rectangular projection object;

A procedure for detecting positions of four corners of the rectangular projection object on an image captured by the imaging device;

A procedure for detecting positions of four corners of the projected test pattern on an image captured by the imaging device;

Based on the relative positional relationship between the four corners of the rectangular projection object and the four corners of the test pattern on the image captured by the imaging device, the spatial light modulation unit is set. Determining the four corners of the virtual projection frame

A combination characterized by causing the computer to execute.