JP2018121201A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To project a corrected projected image as large as possible in a screen.SOLUTION: A CPU (110) determines a target region in which an input image is arranged on the basis of the captured image obtained by imaging the screen direction. A layout processing unit (320) places an input image in the target area while maintaining an aspect ratio of the image. An transformation processing unit (330) applies transformation processing to the image arranged in the target area. The CPU (110) obtains an overlapping area between a screen area and a projection area on which the image projection is performed from the captured image, and determines the target area on the basis of the aspect ratio of the rectangle that can be taken in the overlapping area. Then, in the transformation processing unit (330), deformation processing is performed on the input image such that the target area is projected within the overlapping area.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for processing an image to be projected.

  In recent years, projection display devices such as liquid crystal projectors are widely used for business purposes such as presentations and conferences, and home uses such as home theaters, and accordingly, miniaturization and weight reduction are progressing. Also, there are various places where the projector is installed, and it is often impossible to place the projector in front of the screen due to restrictions on the place. As a general arrangement example, the projector is often installed on a desk or the like with a slight inclination toward the upper screen. Accordingly, an image is projected on the screen in this case from a slightly obliquely downward direction.

  However, in this case, geometric distortion called trapezoidal distortion occurs in the projected image on the screen due to the relative inclination between the projector body and the screen. In order to solve this problem, many general projectors have a keystone correction function (keystone correction function) that corrects keystone distortion by signal processing. In other words, this projector forms an image deformed by the keystone correction on the projection panel surface, and projects the image formed on the panel surface onto the screen, thereby projecting a rectangular image on the screen.

  Further, for example, Patent Document 1 describes a method of calculating a screen aspect ratio based on an image captured by an imaging apparatus and performing keystone correction in accordance with the screen while maintaining the aspect ratio of the input image signal. Yes. In the method described in this document, the screen is imaged by the imaging device in a state where the calibration image is projected on the screen, and the aspect ratio of the screen and the distortion of the image are calculated from the captured image, and based on them. Keystone correction processing is performed.

JP 2008-251026 A

  In the technique described in Patent Document 1, a trapezoidal correction process is performed in which a deformation process is performed so that a projected image fits within a screen. Therefore, if the aspect ratio of the projection panel provided in the projector is different from the aspect ratio of the screen, the projected image after keystone correction will be excessively small relative to the screen depending on the aspect ratio of the input image signal. May end up.

  Therefore, an object of the present invention is to make it possible to project a projection image after deformation processing as large as possible on a screen.

  The present invention is an image processing apparatus that processes an image projected on a screen, a determination unit that determines a target region in which an input image is to be arranged based on a captured image obtained by capturing the screen direction, and the input image The determination unit includes an arrangement unit that arranges the input image in the target area while maintaining an aspect ratio, and a deformation unit that performs a predetermined deformation process on the image arranged in the target area. The means obtains an overlapping area between the area of the screen and a projection area where video projection is performed from the captured image, determines the target area based on a rectangular aspect ratio that can be taken within the overlapping area, and The deformation means performs the deformation process on the input image so that the target area is projected into the overlap area.

  According to the present invention, it is possible to project the projected image after the deformation process as large as possible on the screen.

It is a figure which shows the whole structure of a liquid crystal projector. It is a flowchart which shows the flow of the basic operation | movement of a liquid crystal projector. It is explanatory drawing of how to obtain | require a keystone correction | amendment shape. It is explanatory drawing of the example from which a part of screen deviates from a projection image | video. It is a figure which shows the internal structure of an image process part. It is explanatory drawing of projective transformation. It is a flowchart which shows the flow of the process which correct | amends the trapezoid distortion of a projection image | video. It is explanatory drawing of the image process which correct | amends distortion of a projection image | video, and the distortion. It is explanatory drawing of a projection aspect setting. It is explanatory drawing of the projection shape before and behind correction | amendment, and arrangement | positioning on a liquid crystal panel.

DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
In the present embodiment, a projection display device as an application example of an image processing device, for example, a projector using a transmissive liquid crystal panel is given. The image processing apparatus according to the present embodiment is not limited to a projector using a transmissive liquid crystal panel, but may be any projector such as a projector using DLP (registered trademark) or a reflective liquid crystal panel such as LCOS (registered trademark). Even if it is a thing, it is applicable. Further, a single-plate type, a three-plate type, and the like are generally known as liquid crystal projectors, but the projector of this embodiment may be any type. The liquid crystal projector of this embodiment controls the light transmittance of each pixel in the liquid crystal panel for projection according to the image to be displayed, and projects light from the light source that has passed through the liquid crystal panel onto the screen. Present the video to the user.
Hereinafter, the liquid crystal projector of this embodiment will be described.

<Overall configuration>
FIG. 1 is a diagram illustrating an overall configuration example of a liquid crystal projector 100 that is an application example of the image processing apparatus of the present embodiment.
The liquid crystal projector 100 according to the present embodiment includes a CPU 110, a ROM 111, a RAM 112, an operation unit 113, an image input unit 130, and an image processing unit 140. The liquid crystal projector 100 further includes a liquid crystal control unit 150, liquid crystal panels 151R, 151G, and 151B, a light source control unit 160, a light source 161, a color separation unit 162, a color composition unit 163, an optical system control unit 170, and a projection optical system 171. . In addition, the liquid crystal projector 100 may include a recording / playback unit 191, a recording medium 192, a communication unit 193, an imaging unit 194, a display control unit 195, and a display unit 196.

  CPU 110 controls each block of liquid crystal projector 100. The ROM 111 stores a control program that describes the processing procedure of the CPU 110. The RAM 112 temporarily stores control programs and data as work memory. Further, the CPU 110 reads still image data and moving image data from the recording medium 192 via the recording / reproducing unit 191 and temporarily stores the data in the RAM 112. Further, the CPU 110 temporarily stores the still image and moving image data received via the communication unit 193 in the RAM 112. In addition, the CPU 110 temporarily stores still image data and moving image data captured by the imaging unit 194 in the RAM 112. Further, the CPU 110 executes a program stored in the ROM 111 to convert still image data or moving image data temporarily stored in the RAM 112 into data for recording, and stores it in the recording medium 192 via the recording / playback unit 191. It can also be recorded. In addition, the CPU 110 executes the program stored in the ROM 111 to convert the still image data and moving image data stored in the RAM 112 into display data by the image processing unit 140 and display the data via the display control unit 195. It can also be displayed on the part 196.

  The operation unit 113 receives an instruction from the user and sends the instruction signal to the CPU 110. The operation unit 113 includes, for example, a switch, a dial, a touch panel provided on the display unit 196, and the like. The CPU 110 controls each block of the liquid crystal projector 100 based on the instruction signal input from the operation unit 113. The operation unit 113 may be, for example, a signal receiving unit (infrared receiving unit) that receives a signal (infrared signal) from a remote controller (hereinafter referred to as a remote controller). In this case, the operation unit 113 receives a user instruction signal by infrared rays from the remote controller and sends the instruction signal to the CPU 110. CPU 110 controls each block of liquid crystal projector 100 based on the instruction signal.

  The image input unit 130 is an interface unit for receiving an image signal transmitted from an external device (not shown). Note that the input image signal may be either a still image or a moving image signal. The image input unit 130 includes various input terminals such as a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, an HDMI (registered trademark) terminal, and a USB terminal. Is also included. When receiving an analog image signal, the image input unit 130 converts the analog image signal into a digital image signal and sends the digital image signal to the image processing unit 140. When receiving the digital image signal, the image input unit 130 receives the digital image signal. The signal is sent to the image processing unit 140. Hereinafter, the digital image signal is referred to as image data. Here, the external device may be any device such as a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, or a game machine as long as it can output an image signal.

  The image processing unit 140 subjects the image data received from the image input unit 130 to frame number change processing, pixel number change processing, image shape change processing, and the like, and sends the image data to the liquid crystal control unit 150. The image processing unit 140 can also perform various processes such as a frame thinning process, a frame interpolation process, a resolution conversion process, a pixel analysis process, a predetermined deformation process, a gain process, an offset process, and a panel process. The predetermined deformation process includes a geometric correction process (keystone correction process, curved surface correction process, etc.). The image processing unit 140 also displays each image such as a still image or moving image read from the recording medium 192, a still image or moving image received via the communication unit 193, or a still image or moving image captured by the imaging unit 194. The above-described processes can be performed on the data. The image processing unit 140 is composed of, for example, a microprocessor for image processing. Note that the image processing unit 140 does not need to be provided as a dedicated microprocessor. For example, the CPU 110 may execute the same processing as the image processing unit 140 based on a program stored in the ROM 111.

  The light source control unit 160 controls the on / off of the light source 161 and the amount of light, and is configured by a control microprocessor. The light source control unit 160 does not need to be provided as a dedicated microprocessor. For example, the CPU 110 may execute the same control as the light source control unit 160 based on a program stored in the ROM 111. The light source 161 generates light for projecting an image on a screen (not shown), and may be, for example, a halogen lamp, a xenon lamp, a high-pressure mercury lamp, or the like. The color separation unit 162 separates light from the light source 161 into light of three primary colors of red, green, and blue (hereinafter referred to as R, G, and B). For example, from the dichroic mirror and the prism Become. In addition, when using LED etc. corresponding to each color of R, G, B as the light source 161, the color separation part 162 is unnecessary. The light of each color of R, G, and B separated from the light of the light source 161 by the color separation unit 162 is incident on the liquid crystal panels 151R, 151G, and 151B provided corresponding to the colors of R, G, and B.

  The liquid crystal panels 151R, 151G, and 151B form an image on the panel surface based on the drive signal supplied from the liquid crystal control unit 150. Image formation on the panel surface is performed by changing the amount of transmitted incident light for each pixel (dot). The liquid crystal panels 151R, 151G, and 151B are light modulation panels that perform light modulation for each pixel with respect to incident light of corresponding colors R, G, and B, respectively. In this embodiment, the light modulation panel is a transmissive panel, but may be a reflective panel. The liquid crystal panel 151 </ b> R is a liquid crystal panel provided corresponding to the R light among the light separated into R, G, and B colors by the color separation unit 162. The liquid crystal panel 151R is configured by two-dimensionally arranging a plurality of pixels, and adjusts the transmission amount of R light for each pixel. The liquid crystal panel 151 </ b> G is a liquid crystal panel provided corresponding to the G light among the lights separated into the R, G, and B colors by the color separation unit 162. The liquid crystal panel 151G is configured by two-dimensionally arranging a plurality of pixels, and adjusts the transmission amount of G light for each pixel. The liquid crystal panel 151 </ b> B is a liquid crystal panel provided corresponding to the B light among the lights separated into the R, G, and B colors by the color separation unit 162. The liquid crystal panel 151B is configured by two-dimensionally arranging a plurality of pixels, and adjusts the transmission amount of B light for each pixel.

  The liquid crystal control unit 150 generates a drive signal for adjusting the light transmittance in each pixel of the liquid crystal panels 151R, 151G, and 151B based on the image data of each frame processed by the image processing unit 140. Specifically, the drive signal generated by the liquid crystal control unit 150 is a signal that controls the voltage applied to the liquid crystal of each pixel of the liquid crystal panels 151R, 151G, and 151B. The liquid crystal control unit 150 applies to each pixel of the liquid crystal panels 151R, 151G, and 151B according to the gradation level for each pixel of the R, G, and B color components of each frame image sent from the image processing unit 140. By controlling the voltage to be adjusted, the light transmittance of each pixel is adjusted. The liquid crystal control unit 150 is configured by, for example, a control microprocessor. The liquid crystal control unit 150 does not need to be provided as a dedicated microprocessor. For example, the CPU 110 may execute the same control as the liquid crystal control unit 150 based on a program stored in the ROM 111.

  The optical system control unit 170 controls the projection optical system 171 and includes, for example, a control microprocessor. The optical system control unit 170 does not need to be provided as a dedicated microprocessor. For example, the CPU 110 may execute the same control as the optical system control unit 170 based on a program stored in the ROM 111. The color synthesizing unit 163 synthesizes R, G, and B lights respectively transmitted through the liquid crystal panels 151R, 151G, and 151B, and includes, for example, a dichroic mirror and a prism. The light obtained by combining the R, G, and B lights by the color combining unit 163 enters the projection optical system 171. The projection optical system 171 is an optical system for forming a projected image by the light combined by the color combining unit 163 on the screen. The projection optical system 171 includes a plurality of lenses and lens driving actuators, and by driving the lenses with the actuators, the projection image can be enlarged, reduced, focused, lens shifted, and the like. In the present embodiment, in the liquid crystal panels 151R, 151G, and 151B, the transmittance of each pixel is adjusted by the liquid crystal control unit 150 based on the image data from the image processing unit 140. The color combining unit 163 combines the R, G, and B lights via the liquid crystal panels 151R, 151G, and 151B, and the combined light is projected onto the screen by the projection optical system 171. Therefore, an image corresponding to the image data from the image processing unit 140 is projected and displayed on the screen.

  The recording / playback unit 191 plays back still image data and moving image data from the recording medium 192. The recording / playback unit 191 records still image data and moving image data obtained by the imaging unit 194 on the recording medium 192. The recording / playback unit 191 can also record still image data and moving image data received from the communication unit 193 on the recording medium 192. The recording / playback unit 191 includes, for example, an interface or a microprocessor for recording or playing back image data on the recording medium 192. Note that the recording / reproducing unit 191 does not need to have a dedicated microprocessor. For example, the CPU 110 may execute the same processing as the recording / reproducing unit 191 based on a program stored in the ROM 111. The recording medium 192 can record still image data and moving image data as well as control data necessary for the liquid crystal projector of this embodiment. The recording medium 192 may be any type of recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, and may be a detachable recording medium or a built-in recording medium.

  The communication unit 193 receives control signals, still image data, moving image data, and the like from an external device. The communication unit 193 may be compliant with, for example, a wireless LAN, a wired LAN, USB, Bluetooth (registered trademark), and the like, and is not limited to a specific communication method. When the terminal of the image input unit 130 is, for example, an HDMI (registered trademark) terminal, the communication unit 193 may perform CEC communication via the terminal. The external device may be any device such as a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, a game machine, or a remote controller as long as it can communicate with the liquid crystal projector 100.

  The imaging unit 194 captures an image of the periphery of the liquid crystal projector 100 according to the present embodiment and acquires an image, and captures an imaging range including a video projected on the screen via the projection optical system 171 and the periphery of the screen. It is possible (shooting the screen direction). The imaging unit 194 images a lens (imaging optical system) that forms an optical image of a subject on the imaging surface of the imaging device, an actuator that drives the lens, a microprocessor that controls the actuator, and an optical image that is formed on the imaging surface. It includes an image sensor that converts signals. The imaging unit 194 also includes an AD conversion unit that converts an analog image signal obtained by the imaging element into digital image data. The imaging unit 194 sends the captured image data to the CPU 110, and the CPU 110 temporarily stores the image data from the imaging unit 194 in the RAM 112. Then, the CPU 110 converts the image data temporarily stored in the RAM 112 into data handled by the liquid crystal projector 100 by executing a program stored in the ROM 111. Note that the imaging unit 194 may not only shoot the screen direction but also be able to shoot the viewer side in the opposite direction to the screen, for example.

  The display control unit 195 performs control for causing the display unit 196 to display an image such as an operation screen for operating the liquid crystal projector 100 and a switch icon, and includes a microprocessor that performs display control. The display control unit 195 does not need to be provided as a dedicated microprocessor. For example, the CPU 110 may execute the same processing as the display control unit 195 based on a program stored in the ROM 111. The display unit 196 displays an operation screen and a switch icon for operating the liquid crystal projector 100 under the control of the display control unit 195. The display unit 196 may be anything as long as it can display an image. For example, it may be a liquid crystal display, a CRT display, an organic EL display, or an LED display. In addition, the display unit 196 may emit an LED or the like corresponding to each button in order to post a specific button so that the user can recognize it.

  Note that the image processing unit 140, the liquid crystal control unit 150, the light source control unit 160, the optical system control unit 170, the recording / playback unit 191 and the display control unit 195 of the present embodiment perform the same processing as those of these blocks. There may be one or more possible microprocessors. Or for example, based on the program memorize | stored in ROM111, CPU110 may perform the process similar to each block.

<Basic operation>
FIG. 2 is a flowchart for explaining the basic operation of the liquid crystal projector 100 of the present embodiment. The operation of the flowchart shown in FIG. 2 is basically executed by the CPU 110 controlling each block based on a program stored in the ROM 111. The processing of the flowchart of FIG. 2 is started when the user instructs the liquid crystal projector 100 to be turned on by the operation unit 113 or a remote controller (not shown). In the following description, steps S201 to S210 in the flowchart of FIG. 2 are abbreviated as S201 to S210, and this also applies to other flowcharts described later.

  When the user instructs the liquid crystal projector 100 to be turned on by the operation unit 113 or a remote controller (not shown), the CPU 110 supplies power to each unit of the liquid crystal projector 100 from a power supply circuit of the power supply unit (not shown). In addition, the CPU 110 executes a projection start process as the process of S201. Specifically, the CPU 110 performs lighting control of the light source 161 via an instruction to the light source controller 160 and driving of the liquid crystal panels 151R, 151G, and 151B via an instruction to the liquid crystal controller 150 as the projection start process of S201. Control and operation setting of the image processing unit 140 are performed. In the following description, the liquid crystal panels 151R, 151G, and 151B are referred to as the liquid crystal panel 151 without being distinguished from each other.

  Next, in S202, the CPU 110 determines whether or not the input signal from the image input unit 130 has changed. If the CPU 110 determines in S202 that the input signal has not changed (No), the CPU 110 proceeds to S204. If the CPU 110 determines that the input signal has changed (Yes), the CPU 110 proceeds to S203.

  In step S203, the CPU 110 sends the image signal input from the image input unit 130 to the image processing unit 140, and controls the image processing unit 140 to perform input switching processing. At this time, the image processing unit 140 detects the resolution, frame rate, and the like of the input image signal, samples the input image at a timing suitable for the detection, performs necessary image processing, and then performs the liquid crystal control unit 150. Output to. Thus, an image generated based on the image signal input from the image input unit 130 is formed on the panel surface of the liquid crystal panel 151, and video projection is performed. After S203, the CPU 110 advances the process to S204.

  In step S <b> 204, the CPU 110 determines whether a user operation is input via the operation unit 113. If the CPU 110 determines in S204 that no user operation has been input (No), the process proceeds to S208. If the CPU 110 determines that a user operation has been input (Yes), the CPU 110 proceeds to S205.

  In step S205, the CPU 110 determines whether the user operation input via the operation unit 113 is an end operation. If the CPU 110 determines in S205 that the user operation is an end operation (Yes), the CPU 110 advances the process to S206 and executes the projection end process. Specifically, the CPU 110 performs, as projection termination processing, a light source 161 turn-off control via an instruction to the light source control unit 160, a drive stop control of the liquid crystal panel 151 via an instruction to the liquid crystal control unit 150, and necessary settings. Is stored in the ROM 111. After executing the projection end process of S206, the CPU 110 ends the process of the basic operation flowchart of FIG. On the other hand, if it is determined in S205 that the user operation is not an end operation (No), the CPU 110 advances the process to S207.

  In step S207, the CPU 110 executes user processing corresponding to the content of the user operation input via the operation unit 113. For example, the CPU 110 performs changes in installation settings, input signals, image processing, information display control, and the like as user processing. After S207, the CPU 110 advances the process to S208.

  In step S208, the CPU 110 determines whether the communication unit 193 has received a command. CPU110 returns a process to S202, when it determines with the command not being received in S208 (No). On the other hand, if it is determined in S208 that the command has been received (Yes), the CPU 110 advances the process to S209.

  In step S209, the CPU 110 determines whether the command received by the communication unit 193 is an end command. If the CPU 110 determines in S209 that the command is an end command (Yes), the CPU 110 advances the process to S206 and executes the above-described projection end process. On the other hand, if it is determined in S209 that the command is not an end command (No), the CPU 110 advances the process to S210.

  In step S210, the CPU 110 executes command processing corresponding to the content of the received command. For example, the CPU 110 performs installation setting, input signal setting, image processing setting, status acquisition, and the like as command processing. After S210, the CPU 110 returns the process to S202.

  In the liquid crystal projector 100 of the present embodiment, in addition to the video input from the image input unit 130, still image data or video data read from the recording medium 192 by the recording / playback unit 191 can be displayed. . The liquid crystal projector 100 can also display still image data and video data received from the communication unit 193.

<Overview of keystone correction by keystone correction>
Hereinafter, as a comparative example for a correction process to be described later performed in the liquid crystal projector 100 according to the present embodiment, the existing processes described in Patent Document 1 and the like described above with reference to FIGS. 3A to 3C and FIG. The trapezoid correction process and its problems will be described.
FIG. 3A is a diagram showing a state in which an uncorrected image that has not been subjected to keystone correction processing is projected obliquely downward on a screen 910 having an aspect ratio of, for example, 4: 3. In the case of the example of FIG. 3A, the projected image 920 is an image that is obliquely distorted because the keystone correction processing is not performed.

  Here, the aspect ratio of the screen 910 illustrated in FIG. 3A is 4: 3, while the aspect ratio of the liquid crystal panel for projection of the projector is, for example, 16:10. FIG. 3B shows a state in which an image (hereinafter referred to as a corrected image 930) after being subjected to correction processing that matches the aspect ratio of the screen 910 is projected using the method described in Patent Document 1 and the like. FIG. FIG. 3B also shows a projected image 920 that has not been subjected to correction processing (hereinafter referred to as an uncorrected image 920) for comparison. As shown in FIG. 3B, the corrected image 930 has been subjected to correction processing so as to enter the screen 910. In the example of FIG. 3B, the aspect ratio (4: 3) of the screen 910 is different from the aspect ratio (16:10) of the liquid crystal panel for projection. For this reason, when realizing the correction process with the aspect ratio maintained, the four vertices indicated by the black circles B1 to B4 in the uncorrected image 920 in the figure and the white circles W1 to W4 in the corrected image 930 respectively corresponding to the four vertices. Keystone correction is performed to move to the four vertices indicated by. As a result, as shown in FIG. 3B, areas where the corrected image 930 is not projected are formed above and below the screen 910. Note that, depending on the combination of the aspect ratio of the screen 910 and the aspect ratio of the liquid crystal panel for projection, there may be a region where an image is not projected on the left and right of the screen 910.

  FIG. 3C shows a keystone correction process for an input image signal having an aspect ratio of 4: 3 when the aspect ratio of the projection liquid crystal panel is 16:10 and the aspect ratio of the screen 910 is 4: 3. It is a figure showing the state which projected the image after performing. In this example, since the aspect ratio of the input image signal is 4: 3, the corrected video 940 after the keystone correction is performed on the image signal is indicated by the diagonal lines in FIG. It will be projected as an image of the size shown. That is, when the method of Patent Document 1 is used, the corrected video 940 is excessive with respect to the screen 910 even though the aspect ratio of the screen 910 and the aspect ratio of the input image signal are both equal (4: 3). It is made small and projected.

  In addition, since an image is projected using the entire panel surface in the projection liquid crystal panel, the corrected image 930 is always a projection image included in the projection area where the uncorrected image 920 was projected. On the other hand, depending on how the liquid crystal projector 100 is installed, for example, as shown in FIG. 4, a portion where a projected image (uncorrected image 920 is shown in this example) is not projected on a part of the screen 910. There is also. In this case, even if the correction processing according to the aspect ratio of the screen 910 is performed as in the method described in Patent Document 1, the image is still projected on the portion where the uncorrected image 920 is not projected on the screen 910. The screen 910 cannot be used effectively.

<Description of Configuration and Operation of Image Processing Unit of First Embodiment>
For this reason, the liquid crystal projector 100 according to the present embodiment includes the image processing unit 140 illustrated in FIG. 5, and performs a process described below to display a corrected video while maintaining the aspect ratio of the input image signal. It is possible to project as large as possible on the screen.
FIG. 5 is a block diagram illustrating a detailed configuration of the image processing unit 140 of the liquid crystal projector 100 according to the present embodiment. The image processing unit 140 according to the present embodiment includes various image processing units 310, a layout processing unit 320, and a deformation processing unit 330. 5 also shows the CPU 110, the liquid crystal control unit 150, and the frame memory 340. The frame memory 340 is a memory for the deformation processing unit 330 to store an image before or after the keystone correction, and is included in the RAM 112, for example.

  In FIG. 5, an original image signal 3010 is an image signal input from the above-described image input unit 130, recording / reproducing unit 191, communication unit 193, and the like. The timing signal 3020 is a timing signal such as a vertical synchronization signal, a horizontal synchronization signal, and a clock synchronized with the original image signal 3010, and is supplied from the source of the original image signal 3010. Each block in the image processing unit 140 operates based on the timing signal 3020. Note that the image processing unit 140 may be configured to regenerate a timing signal necessary for each block from the timing signal 3020.

  The various image processing units 310 perform statistical processing on the original image signal 3010 in cooperation with the CPU 110, and acquire statistical information including a histogram and APL (Average_Picture_Level). Various image processing units 310 perform various image processing on the original image signal 3010. Various image processing units 310 output post-image processing signals 3030 that are image signals after various image processings to the layout processing unit 320. Examples of various types of image processing include IP conversion, frame rate conversion, resolution conversion, γ conversion, color gamut conversion, gain processing, color correction, edge enhancement, and the like. Since there is, the description is omitted. The statistical information is used for gain processing or the like, but the description thereof is also omitted.

The layout processing unit 320 performs layout processing for determining the arrangement on the liquid crystal panel 151 with respect to the post-image processing signal 3030 based on an instruction from the CPU 110. Although details will be described later, the CPU 110 obtains the projection target area and its aspect ratio, and controls the layout processing in the layout processing unit 320 based on the projection target area and the aspect ratio. The layout processing unit 320 outputs a post-layout signal 3040 obtained by performing layout processing on the post-image processing signal 3030 based on an instruction from the CPU 110 to the deformation processing unit 330. The layout processing unit 320 can also perform processing such as so-called OSD (on-screen display) composition. A specific example of OSD will be described later.
Note that the order of the various image processing units 310 and the layout processing unit 320 is not limited to the example of FIG. 5, and may be configured to perform various image processing after the layout processing.

The deformation processing unit 330 performs predetermined deformation processing based on the following equations (1) and (2), that is, keystone correction processing, on the image of the post-layout signal 3040. Then, the deformation processing unit 330 outputs the image signal after the deformation processing (hereinafter referred to as a post-deformation image signal 3050) to the liquid crystal control unit 150.
Here, the keystone correction can be realized by projective transformation. If the coordinates of an arbitrary pixel of the original image before the correction process are (xs, ys), the coordinates (xd, yd) after the deformation process corresponding to the pixel are set. Is represented by equation (1).

  M in Expression (1) is a 3 × 3 matrix, which is a projective transformation matrix from the original image to the image after the deformation process. (Xso, yso) is the coordinates of one vertex of the original image 800 indicated by the solid line in FIG. 6, and (xdo, ydo) is each vertex of the image 810 after the deformation process indicated by the one-dot chain line in FIG. Among the coordinate values of the vertices corresponding to the vertices (xso, yso) of the original image 800.

In addition, the transformation processing unit 330 receives the inverse matrix M ⁻¹ of the matrix M of Equation (1) and the offset information from the CPU 110, and the coordinate values (xd, yd) of the image 810 after transformation processing according to Equation (2). ) To obtain the coordinates (xs, ys) of the original image corresponding to. The offset information is a coordinate value of (xso, yso) at each vertex of the original image 800 and a coordinate value of (xdo, ydo) at each vertex of the image 810 after the deformation process.

  Here, if the coordinates of the original image obtained from Equation (2) become an integer, the pixel value of the coordinates (xs, ys) of the original image is used as it is as the coordinates (xd, yd) of the image 810 after the transformation processing. It is good also as a pixel value. However, since the coordinates of the original image obtained by Expression (2) are not always integers, when the coordinates of the original image obtained by Expression (2) do not become integers, the deformation processing unit 330 calculates the values of peripheral pixels. The pixel value of the coordinates (xd, yd) of the image 810 after the deformation process is obtained by the interpolation process using. The interpolation method may be bilinear, bicubic, or any other interpolation method. When the coordinates of the original image obtained from Expression (2) are outside the range of the area of the original image 800, the deformation processing unit 330 sets the pixel value to the value of black or the background color set by the user. .

The deformation processing unit 330 obtains the pixel values of all coordinates after the deformation processing as described above, generates a post-deformation image, and outputs the post-deformation image signal 3050 to the liquid crystal control unit 150. As a result, the image after the deformation process is displayed on the panel surface of the liquid crystal panel 151.
In the above description, the information of the inverse matrix M ⁻¹ of the matrix M is input from the CPU 110 to the image processing unit 140, but only the information of the matrix M is input to the image processing unit 140. An inverse matrix M ⁻¹ may be obtained.

Hereinafter, in the liquid crystal projector 100 of the present embodiment, a method for projecting a video image as large as possible on the screen when the image after the deformation process as described above is formed on the liquid crystal panel 151 will be described with reference to FIGS. While explaining.
FIG. 7 is a flowchart showing the flow of processing executed by the CPU 110 controlling each block based on the program according to the present embodiment stored in the ROM 111. The process of the flowchart in FIG. 7 starts when the user selects the execution start of the automatic correction process of the keystone by the operation unit 113 or a remote controller (not shown). The processing of the flowchart of FIG. 7 is performed by the automatic correction function of the keystone by detecting that the installation location of the main body of the liquid crystal projector 100 is moved or the inclination is changed by a vibration sensor (not shown). It may start when it is activated.

  When the CPU 110 starts the keystone automatic correction process, first, in step S401, the CPU 110 controls the liquid crystal control unit 150 to form a black image on the liquid crystal panel 151. As a result, a black image is projected from the projection optical system 171 in the screen direction.

  Next, in step S <b> 402, the CPU 110 controls the imaging unit 194 to image the screen direction. Note that the imaging range of the imaging unit 194 is a range that includes not only the entire surface of the screen but also its periphery. Therefore, the imaging unit 194 images the screen and its surroundings. The captured image data acquired by the imaging unit 194 is stored in the RAM 112 under the control of the CPU 110. The reason why the black image is projected in the screen direction and the screen and its surroundings are imaged is to reduce the influence of the projected image and facilitate the screen area detection process performed in S403 described later. In the present embodiment, a black image is projected. However, the influence of the projected image may be reduced by performing control to reduce the light amount of the light source 161.

  Here, FIG. 8B shows the relative relationship between the imaging unit 194 of the liquid crystal projector 100 and the screen 530 when the liquid crystal projector 100 is installed obliquely without facing the screen 530. It is the figure which showed the positional relationship. Note that a rectangular area 540 in FIG. 8B schematically represents an imaging surface (hereinafter, referred to as an imaging surface 540) of an imaging element provided in the imaging unit 194. The imaging surface 540 is a surface perpendicular to the optical axis z of the imaging optical system of the imaging unit 194. The optical axis z of the imaging unit 194 is an axis parallel to the optical axis of the projection optical system 171. On the other hand, in the example of FIG. 8B, the liquid crystal projector 100 is installed obliquely without facing the screen 530, so the screen 530 is inclined with respect to the optical axis z.

  FIG. 8A shows an example of a captured image 510 obtained by imaging the screen direction in a state in which an uncorrected black image is projected in the example of the positional relationship shown in FIG. It is the figure which showed an example. The captured image 510 in FIG. 8A includes an image area of the screen 530 (hereinafter, referred to as a screen area 520) captured by the imaging unit 194 in FIG. That is, when the liquid crystal projector 100 is installed obliquely with respect to the screen 530 as shown in FIG. 8B, the screen 530 that is actually rectangular is the screen area 520 of the captured image 510 in FIG. The image is captured as a distorted quadrilateral image.

  When the captured image is acquired in S402 as described above, the CPU 110 next detects the screen area 520 from the captured image 510 captured in S402 and stored in the RAM 112 in S403. When the liquid crystal projector 100 is installed obliquely with respect to the screen 530 as shown in FIG. 8B, in S403, the CPU 110 determines from the captured image 510 of FIG. Will be detected. In the present embodiment, the CPU 110 detects the screen area 520 by obtaining four sides or four vertices of the screen image from the captured image 510. More specifically, the CPU 110 converts the captured image 510 into a binary image, and performs a Hough transform on each pixel point of the converted binary image to detect a line segment. The four sides of the screen area 520 are detected. Further, the CPU 110 can detect the points where the sides of the screen area 520 intersect as four vertices.

  Next, in S404, the CPU 110 calculates the three-dimensional coordinates of the screen 530 using the liquid crystal projector 100 (specifically, the imaging unit 194) as the reference coordinates RC. Here, since the imaging unit 194 of the liquid crystal projector 100 images the screen 530 and its surroundings, the image of the screen 530 is mapped on the imaging surface 540 of the imaging unit 194 as shown in FIG. (Hereinafter referred to as screen mapping 541) is formed. Therefore, the vertices P1 ′ to P4 ′ of the screen map 541 on the imaging surface 540 correspond to the vertices P1 to P4 of the screen 530, and further, the screen area 520 in the captured image 510 of FIG. Each vertex P1 ′ to P4 ′ also corresponds. That is, the xy coordinates of the four vertices P1 ′ to P4 ′ of the screen area 520 detected in S403 correspond to the xy coordinates of the vertices P1 ′ to P4 ′ of the screen mapping 541 on the imaging surface 540 of the imaging unit 194, respectively. doing. Further, the three-dimensional coordinates of the vertices P1 to P4 of the screen 530 are coordinates on an extension line of a straight line passing through the vertices P1 ′ to P4 ′ of the screen mapping 541 from the reference coordinate RC. Since the coordinates of each pixel on the imaging surface 540 of the imaging unit 194 are known, the xy coordinates of the vertices P1 ′ to P4 ′ of the screen mapping 541 are also known. Therefore, if the distances z1 to z4 from the reference coordinate RC to the vertices P1 to P4 of the screen 530 are found on the extended line of each straight line passing through the vertices P1 ′ to P4 ′ of the screen mapping 541 from the reference coordinates RC, Each three-dimensional coordinate of each vertex P1-P4 can be calculated.

In the present embodiment, since the actual shape of the screen 530 is a rectangle, the lengths of the sides facing each other are equal, and the lengths of the diagonal lines are also equal. That is, the distance between the vertices of the screen 530 has a relationship represented by the following formulas (3) to (5).
(Distance between P1 and P2) = (Distance between P3 and P4) Equation (3)
(Distance between P1 and P4) = (Distance between P2 and P3) Equation (4)
(Distance between P1 and P3) = (Distance between P2 and P4) Equation (5)
Here, when calculating the three-dimensional coordinates of the screen 530, among these equations (3) to (5) and the four unknowns (each distance z1 to z4), each distance z1 to z4 is an actual distance. You don't need a value, you just know the relative value of each. For this reason, if the value of one of the distances z1 to z4 is assumed to be an arbitrary value, the three-dimensional coordinates of the vertices P1 to P4 of the screen 530 are based on the expressions (3) to (5). Can be calculated. In step S404, the CPU 110 calculates the three-dimensional coordinates of the screen 530 in this way.

  Next, in step S405, the CPU 110 controls the liquid crystal control unit 150 to cause the liquid crystal panel 151 to form a white image that has not been subjected to keystone correction. As a result, a white image is projected from the projection optical system 171 in the screen direction. In step S406, the CPU 110 controls the imaging unit 194 to capture an image in the screen direction. Data of the captured image acquired by the imaging unit 194 is stored in the RAM 112 under the control of the CPU 110.

Next, in S407, the CPU 110 compares and analyzes the captured image at the time of white projection in S406 and the captured image at the time of black projection in S402, thereby projecting an uncorrected video that has not been subjected to keystone correction. An overlapping area between the projected video area and the screen 530 is detected.
FIG. 8C shows the captured image 511 acquired in S406, and shows an example of how the projected video area 550 of the uncorrected video (in this case, the uncorrected white image) and the screen area 520 overlap. It is a figure. In FIG. 8C, the projected image area 550 of the uncorrected image is shown as a rectangular area for convenience, but this is the case when the outside of the screen 530 is a wall or the like that is substantially flush with the screen. Limited. The region that protrudes from the screen 530 is not limited to this because it depends on the surrounding environment.

  Here, compared with the captured image 510 of FIG. 8A acquired in S402 during black projection, the captured image 511 of FIG. 8C acquired in S406 during white projection is a projected video region 550 of an uncorrected video, That is, the pixel value of the area where the white image is projected is significantly increased. Therefore, the CPU 110 sets the xy coordinates on the imaging surface of the imaging unit 194 at a point where the pixel value changes greatly on the side of the screen area 520 (a point where the change amount of the pixel value is large, hereinafter referred to as a changing point). Ask. In the example of FIG. 8C, on the side of the screen area 520, from the pixel at the vertex P1 ′ to the pixel immediately before the point Q1 ′, and from the pixel at the vertex P1 ′ to the pixel immediately before the point Q2 ′. It is assumed that each pixel value is low up to the pixel and the pixel value hardly changes. On the other hand, on the side of the screen region 520, the pixel value is high from the pixel at the vertex P2 ′ to the pixel at the point Q1 ′, and from the pixel at the vertex P4 ′ to the pixel at the point Q2 ′, and the change in the pixel value Suppose that there is almost no. In this case, the pixel at the point Q1 ′ is a pixel whose amount of change in pixel value is larger than the pixel immediately before the point Q1 ′ on the vertex P1 ′ side. Therefore, the CPU 110 detects the point Q1 ′ on the side of the screen area 520 as the change point Q1 ′, and obtains the xy coordinates of the pixel at the change point Q1 ′. Similarly, the pixel at the point Q2 ′ is a pixel whose amount of change in pixel value is larger than the pixel immediately before the point Q2 ′ on the vertex P1 ′ side. Therefore, the CPU 110 detects the point Q2 ′ on the side of the screen area 520 as the change point Q2 ′, and obtains the xy coordinates of the pixel at the change point Q2 ′.

  The three-dimensional coordinates of the point on the screen 530 (referred to as point Q1) to which the change point Q1 ′ corresponds correspond to the extension of a straight line connecting the reference coordinate RC of the image pickup unit 194 and the change point Q1 ′ on the image pickup surface. This is the coordinates of the intersection of the line and the line segment P1-P2 connecting the vertices P1-P2 of the screen 530. Similarly, the three-dimensional coordinate of the point on the screen 530 (referred to as point Q2) corresponding to the change point Q2 ′ is a straight line connecting the reference coordinate RC of the image pickup unit 194 and the change point Q2 ′ on the image pickup surface. The coordinates of the intersection of the extension line and the line segment P1-P4 connecting the vertices P1 to P4 of the screen 530. FIG. 8D is a diagram showing points Q1 and Q2 on the plane of the screen 530. In this example, a pentagonal region having the points Q1, P2, P3, P4, and Q2 as vertices is a screen 530. And an uncorrected video projection video area 550 is an overlapping area 560. In S407, the CPU 110 detects the overlapping region 560 as described above.

  In FIG. 8C, an example in which a part of the screen area 520 is out of the projected image area 550 of the uncorrected image is given, and thus the overlapping area 560 includes the points Q1, Q1 shown in FIG. It is detected as a pentagonal region having apexes P2, P3, P4 and Q2. On the other hand, although not shown, for example, when the entire screen area is within the projected video area of the uncorrected video, the CPU 110 detects the screen area 520 as an overlapping area.

Next, in S408, the CPU 110 calculates the aspect ratio of the projection target area.
Here, the CPU 110 determines the projection target area as follows. The CPU 110 first obtains a rectangular area based on the overlapping area.
In the case of the example of FIG. 8D, the CPU 110, as rectangular areas that can be taken in the overlapping area 560, is a rectangular area that includes the points Q1, P2, and P3 as vertices and the points Q2, P4, and P3 as vertices. And the rectangular area to include. Then, the CPU 110 employs the rectangular area having the largest area among the rectangular areas that can be taken in the overlapping area 560 as the projection target area, and calculates the aspect ratio of the projection target area. For example, in the case of FIG. 8D, a rectangle including the points Q1, P2, and P3 as vertices is adopted as the projection target area. Then, the CPU 110 calculates the length of the line segment P2-P3 and the length of the line segment Q1-P2 from the three-dimensional coordinates of the vertices P1 to P4 obtained in S404 and the three-dimensional coordinate of the point Q1 obtained in S407. Ask. Further, the CPU 110 calculates the ratio of the length of the line segments P2-P3 and the length of the line segments Q1-P2 as the aspect ratio of the projection target area.

  Note that since the entire screen area is within the projected video area of the uncorrected video, when the screen area 520 is detected as an overlapping area, the CPU 110 is a rectangle based on the three-dimensional coordinates of the vertices P1 to P4 of the screen 530. Is the projection target area. Then, the CPU 110 calculates the aspect ratio of the projection target area, that is, the aspect ratio of the screen 530.

  Next, in S409, the CPU 110 compares the aspect ratio of the projection target area calculated in S408 with the projection aspect ratio set by the user via, for example, a GUI (Graphical User Interface) menu. The projected aspect ratio is an aspect ratio of an area in which an image can be laid out on the panel surface of the liquid crystal panel 151. In the present embodiment, the projected aspect ratio is an aspect ratio of an area in which an image of the post-image processing signal 3030 in FIG. is there. Specific examples of the projection aspect ratio will be described with reference to FIGS. 9A to 9C. The CPU 110 controls resolution conversion in the various image processing units 310 and controls layout processing in the layout processing unit 320 based on the comparison result between the aspect ratio of the projection target area and the projection aspect ratio. Although details will be described with reference to FIGS. 10A to 10F, the CPU 110 converts the original image signal 3010 into the projection aspect ratio and the resolution of the liquid crystal panel 151 to the various image processing units 310 and the layout processing unit 320. In addition, resolution conversion and layout processing are performed.

  FIGS. 9A to 9C are diagrams showing a panel surface of the liquid crystal panel 151, that is, a display area 610 that is an area where an image is formed. The aspect ratio of the display area 610 is 16:10. Suppose that When the projection aspect ratio is set to 16:10, the aspect ratio of the display area 610 of the liquid crystal panel 151 is equal to the aspect ratio. Therefore, as shown in FIG. It matches the display area 610. On the other hand, when the projection aspect ratio is set to be longer than 16:10, areas 630 in which an image can be laid out are areas indicated by diagonal lines above and below the display area 610, as shown in FIG. The area is excluded. An area indicated by diagonal lines above and below the display area 610 is outside the effective area, and, for example, a black image is formed (displayed). When the projection aspect ratio is set to be vertically longer than 16:10, the area 640 where the image can be laid out is an area indicated by diagonal lines on the left and right sides of the display area 610 as shown in FIG. The area is excluded. An area indicated by diagonal lines on the left and right of the display area 610 is outside the effective area, and for example, a black image is formed (displayed).

  In S409, when the CPU 110 determines that the two compared aspect ratios are equal (Yes), the process proceeds to S410. In step S410, the CPU 110 controls the image processing unit 140 to execute the deformation process by the deformation processing unit 330 because an appropriate projection aspect ratio has been set. Details of the deformation process in S410 will be described later. After S410, the CPU 110 ends the process of the flowchart of FIG.

  On the other hand, in S409, if the CPU 110 determines that the two compared aspect ratios are not equal (No), the process proceeds to S411. In step S411, the CPU 110 controls the layout processing unit 320 of the image processing unit 140 to generate an OSD image, and sends it to the liquid crystal panel 151 via the liquid crystal control unit 150 to perform OSD display. In the OSD display at this time, for example, a display change dialog of the projection aspect ratio is displayed. For example, when the aspect ratio of the projection target area obtained in S408 is M: N, but the projection aspect ratio is other than M: N, the CPU 110 determines that “M: N is the optimal projection aspect. Make OSD display of a message such as “Do you want to change it?”. Note that there is a high possibility that the aspect ratio of the projection target area obtained in S408 does not become a general aspect ratio (16: 9, 4: 3, 16:10, etc.), but this is not a problem.

  Next, in S412, the CPU 110 determines the content of the operation input by the user using the operation unit 113 or a remote controller (not shown). For example, if the user operation is a cancel operation, the CPU 110 does not set the projection aspect ratio and advances the process to S410. On the other hand, when the user operation is, for example, a projection aspect ratio setting operation, the CPU 110 advances the process to step S413.

  In step S413, the CPU 110 sets the projection aspect ratio input by the user using the operation unit 113 or the like. Specifically, the CPU 110 changes the aspect ratio of the area where the image can be laid out, that is, the projection aspect ratio, with respect to the layout processing unit 320 of FIG. Along with this, the CPU 110 causes the various image processing units 310 to perform settings for performing appropriate resolution conversion for the projection aspect ratio. After S413, the CPU 110 advances the process to S410.

  In step S410, the CPU 110 sets the coordinates of the four vertices of the original image to be displayed based on the setting of the projection aspect ratio. Specifically, the CPU 110 determines the coordinates of the four vertices of the original image as one of the areas 620 to 640 shown in FIGS. 9A to 9C according to the setting of the projection aspect ratio. Set to the coordinates of the four vertices of the region. In addition, the CPU 110 sets the coordinates of the four vertices of the image after the keystone correction deformation process based on the projection target area determined in S408. Note that, for example, when the imaging optical system and the projection optical system 171 are the same, that is, when the imaging unit 194 is configured to use the projection optical system 171 as the imaging optical system, the CPU 110 performs processing shown in FIG. Three vertex coordinates are obtained from the coordinates of the points Q1 ′, P2 ′, and P3 ′. The vertex coordinates of the remaining one point are obtained from the coordinates mapped on the imaging surface 540 because the corrected projected image becomes a rectangle on the screen 530 and the three-dimensional coordinates are uniquely determined. On the other hand, when the projection optical system 171 and the imaging optical system are prepared separately, coordinate conversion is performed according to the distance between the optical axes of the optical systems, so that the coordinates of the four vertices of the transformed image are obtained. You may ask for. Alternatively, once deformation and projection are performed based on the coordinates of the points Q1 ′, P2 ′, and P3 ′, an error is detected again by imaging using the imaging unit 194, and the error is corrected, The coordinates of the four vertices may be obtained. Alternatively, the coordinates of the four vertices of the image after the deformation process may be calculated based on the keystone correction setting value set from the GUI menu or the like, and the positions of the zoom lens and the shift lens without using the captured image.

FIG. 10A to FIG. 10F are diagrams for explaining the relationship between the arrangement of images in the display area of the liquid crystal panel 151 and the projection state of the projected video.
FIG. 10A is a diagram showing an example of a projection state when the liquid crystal projector 100 is installed obliquely on the screen 530 having an aspect ratio of 4: 3 and an uncorrected video is projected. In the case of the projection state example in FIG. 10A, the uncorrected projected image 710 is partly off the screen 530 and the shape thereof is also distorted.

  FIG. 10B shows an example of a projection state when the aspect ratio of the display area 610 on the liquid crystal panel 151 is 16:10, and the setting of the projection aspect ratio in S413 is changed to the same M: N as the projection target area. FIG. In the case of the projection state example in FIG. 10B, the aspect ratio of the area 640 where the image can be laid out on the liquid crystal panel 151 is M: N. Accordingly, the area indicated by the oblique lines in FIG. 10B is outside the effective area as described above, and, for example, a black image is formed (displayed). In the following examples shown in FIGS. 10C to 10F, it is assumed that a black image is formed (displayed) outside the effective area.

  In FIG. 10C, deformation processing is performed in S410, and the four vertices (black circles B1 to B4 in the figure) of the uncorrected projected image 720 determined by the setting of the projection aspect ratio correspond to the projection target areas, respectively. It is a figure which shows the state matched with 4 vertex (white circle W1-W4 in a figure).

  FIG. 10D is a diagram showing an image shape (deformed shape 730) when an image after deformation processing is formed on the liquid crystal panel 151 in the case of FIG. 10C. Black circles B1 to B4 and white circles W1 to W4 in FIG. 10D represent positions corresponding to the black circles B1 to B4 and white circles W1 to W4 in FIG. The deformed shape 730 may be larger than the area 640 in which an image can be laid out on the liquid crystal panel 151. As a result, when an image having an aspect ratio of M: N is input, the video is projected over the entire projection target area. When an image with another aspect ratio is input, the image is arranged so that either the vertical width or the horizontal width of the projection target area is filled in the area 640 where the image can be laid out on the liquid crystal panel 151. The

  FIG. 10E shows a layout example of an image after deformation processing (deformed shape 730) on the liquid crystal panel 151 when an image having an aspect ratio greater than M: N is input as an image to be displayed. FIG. In the case of FIG. 10E, layout candidates include a layout in which an image is arranged on the upper side and a black band is displayed on the lower side, a layout in which the image is arranged on the lower side and a black band is displayed on the upper side, and an image There is a layout in which a black band is displayed on the upper and lower sides with the center in the center. In the example of FIG. 10 (e), a layout that is closer to the longer one of the upper side and the lower side of the deformed shape 730 (arranged in a position close to the lower side of the longer side in this example) is selected. Is placed.

  FIG. 10F shows a layout example of an image (deformed shape 730) after deformation processing on the liquid crystal panel 151 when an image having an aspect ratio that is longer than M: N is input as an image to be displayed. FIG. In the case of FIG. 10F, layout candidates are a layout in which an image is arranged on the left side and a black band is displayed on the right side, a layout in which an image is arranged on the right side and a black band is displayed on the left side, and the image is the center There is a layout that displays black bands on the left and right sides. In the example of FIG. 10 (f), a layout is selected that is closer to the longer side of the left side and the right side of the deformed shape 730 (in this example, arranged at a position close to the longer left side), and a black belt 750 is placed on the right side. Be placed.

  According to the liquid crystal projector 100 of the present embodiment, as shown in FIGS. 10A to 10F, an image can be displayed using as many pixels on the liquid crystal panel 151 as possible. Display with high image quality is possible.

As described above, according to this embodiment, the input image signal is subjected to resolution conversion and layout processing according to the setting of the projection aspect ratio, and the area according to the projection aspect setting can be taken within the above-described overlapping area. It is transformed to fit the rectangle. As a result, the image after the transformation process by the keystone correction is projected at the maximum size that can be displayed on the screen while maintaining the aspect ratio of the input image signal.
In the present embodiment, when the screen aspect ratio and the projection aspect ratio are different, the setting change dialog is displayed to reflect the user input. However, even if the projection aspect setting is automatically performed without the user input. Good.

<Description of Second Embodiment>
Hereinafter, the second embodiment will be described. Since the overall configuration and basic operation of the liquid crystal projector 100 of the second embodiment are the same as those of the first embodiment described above, description thereof is omitted. In the first embodiment, when determining the projection target region, the rectangle having the larger area is adopted. However, in the second embodiment, among the various input terminals of the image input unit 130, The projection target area is determined according to the selected input terminal (input image signal is input).

  As described above, the image input unit 130 includes various input terminals to which a PC signal and an audio / video signal (AV signal) from a personal computer (PC) are input. Among these input terminals, for example, an analog RGB terminal, a component terminal, an HDMI terminal, a DVI terminal, and the like often input a horizontally long image signal having an aspect ratio of 16: 9 or 4: 3. Therefore, when the input terminal selected (connected) in the image input unit 130 is the analog RGB terminal, the component terminal, the HDMI terminal, the DVI terminal, or the like, the CPU 110 has a horizontally long projection target area. A rectangle is adopted. In addition, when a terminal for inputting image data such as a photograph is selected (connected), such as a USB terminal, the photograph image is often a vertically long image. Therefore, the CPU 110 is the case of the first embodiment. Similarly, the larger area is adopted as the projection target area.

  As described above, in the second embodiment, an appropriate projection target area can be selected by the selected (connected) input terminal, and thus, the projection display on the screen can be performed with the largest possible size. .

<Description of Third Embodiment>
Hereinafter, a third embodiment will be described. Since the overall configuration and basic operation of the liquid crystal projector 100 of the third embodiment are the same as those of the first embodiment described above, description thereof is omitted. In the first embodiment, when an image signal having an aspect ratio different from that of the projection target area is input, the layout is determined so that the pixels of the liquid crystal panel 151 can be used as much as possible. However, in the third embodiment, the projection is performed. The layout is such that an image is arranged closer to the optical axis of the optical system 171.

That is, since the CPU 110 grasps the pixel position corresponding to the optical axis of the projection optical system 171 via the optical system control unit 170, when an image signal having an aspect ratio different from that of the projection target area is input. Then, the layout for arranging the image closer to the optical axis is determined.
Since the optical characteristic of the projection optical system 171 is often better in the central part than in the peripheral part, by arranging an image closer to the optical axis as in the third embodiment, optical distortion and chromatic aberration can be achieved. This makes it possible to project high-quality images with little image quality.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. In the present invention, each function of the above-described embodiment can be realized by cooperation of a circuit (for example, ASIC) and a program.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  100 liquid crystal projector, 110 CPU, 140 image processing unit, 150 liquid crystal control unit, 151R, 151G, 151B liquid crystal panel, 171 projection optical system, 194 imaging unit, 310 various image processing units, 320 layout processing unit, 330 deformation processing unit, 340 frame memory

Claims (12)

An image processing apparatus for processing an image projected on a screen,
A determination means for determining a target area in which an input image is to be arranged based on a captured image obtained by capturing the screen direction;
Arrangement means for arranging the input image in the target area while maintaining the aspect ratio of the input image;
Deformation means for performing a predetermined deformation process on the image arranged in the target area,
The determining means obtains an overlapping area between the screen area and a projection area where video projection is performed from the captured image, and determines the target area based on a rectangular aspect ratio that can be taken within the overlapping area. And
The image processing apparatus, wherein the deformation unit performs the deformation process on the input image so that the target area is projected into the overlap area.   When the aspect ratio of the input image is different from the aspect ratio of the target area, the arrangement unit selects a candidate according to the shape of the screen area from a plurality of candidates for arranging the input image. The image processing apparatus according to claim 1.   When the aspect ratio of the input image is a horizontally long aspect ratio than the aspect ratio of the target area, the arrangement unit is closer to the longer side by comparing the upper side and the lower side of the shape of the screen area. The image processing apparatus according to claim 2, wherein the input image is arranged at a position.   When the aspect ratio of the input image is an aspect ratio that is vertically longer than the aspect ratio of the target area, the arrangement unit compares the left side and the right side of the shape of the screen area and is closer to the longer side The image processing apparatus according to claim 2, wherein the input image is arranged at a position.   5. The image processing apparatus according to claim 1, wherein the arrangement unit further arranges the input image in the target area so as to match an aspect ratio set by a user. 6. .   The determination unit obtains the overlapping region based on the captured image in the screen direction on which the black image is projected and the captured image in the screen direction on which the white image is projected. The image processing apparatus according to any one of 5.   The said determination means determines the said object area | region based on the magnitude | size of the area of the said rectangle, when there are two or more rectangles which can be taken in the said overlap area | region, The any one of Claim 1 to 6 characterized by the above-mentioned. The image processing apparatus according to item.   The image processing apparatus according to claim 7, wherein the determination unit determines the rectangular region having the largest area as the target region.   The said determination means determines the said rectangle which can be taken within the said overlap area | region according to the input terminal into which the signal of the said input image is input, The any one of Claim 1 to 6 characterized by the above-mentioned. Image processing apparatus.   When the aspect ratio of the input image is different from the aspect ratio of the target area, the arrangement means has an intersection of diagonal lines of the input image at a position close to the optical axis of the optical system that projects an image on the screen. The image processing apparatus according to claim 1, wherein the input image is arranged as described above. An image processing method of an image processing apparatus for processing an image projected on a screen,
A determination step of determining a target region in which an input image is to be arranged based on a captured image obtained by capturing the screen direction;
An arrangement step of arranging the input image in the target area while maintaining an aspect ratio of the input image;
A deformation step of performing a predetermined deformation process on the image arranged in the target area,
In the determination step, an overlapping area between the screen area and a projection area on which video projection is performed is obtained from the captured image, and the target area is determined based on a rectangular aspect ratio that can be taken in the overlapping area. ,
In the deformation step, the deformation process is performed on the input image so that the target area is projected into the overlapping area.   The program for functioning a computer as each means of the image processing apparatus of any one of Claim 1 to 10.

Images