JP4379029B2 - Image processing apparatus, image processing method, and image projection apparatus - Google Patents

Description

  The present invention faithfully reproduces the level of a video area that is transformed from a rectangular shape to a non-rectangular shape during distortion correction and the like, and the pixel data levels of other peripheral areas remain constant after the subsequent signal processing. The present invention relates to an image processing apparatus and a method thereof, and an image projection apparatus.

  In an image display apparatus provided with an image display device such as an LCD (liquid crystal display) or a DMD (digital micro-mirror device), for example, the horizontal and vertical sizes of the image display surface of the LCD panel are 16 (horizontal): 9 ( (Vertical) or 4 (horizontal): 3 (vertical), and a certain ratio (hereinafter referred to as aspect ratio). Therefore, in the image display device, when an 4: 3 image is displayed on an LCD panel with an aspect ratio of 16: 9, or conversely, when an 16: 9 image is displayed on an LCD panel with an aspect ratio of 4: 3. In addition, these image sizes are converted using the pixel number conversion function of the built-in image processing apparatus (image processing IC). At this time, it is necessary to set a screen area (hereinafter referred to as a peripheral area) having a certain level of luminance and color around a screen area (hereinafter referred to as a video area) where an image is effectively displayed.

  The conventional image processing apparatus displays the 4: 3 image on the 16: 9 LCD panel, and the LCD panel displays the 16: 9 image on the 4: 3 LCD panel. A band-shaped peripheral area is set above and below the surface, and the peripheral area is fixed to a certain level (usually black level) such as black or gray. This is because when the signal processing for changing the luminance level of the signal such as gain adjustment, bias adjustment, or gamma correction is performed after the aspect ratio conversion, the black level changes, and a phenomenon called so-called black floating occurs. This is a process for preventing such a phenomenon and obtaining a high-quality image.

Also, recent television receivers or computer displays have a multi-screen display in which one screen is divided into a plurality of screens, or a P in P (picture-in-picture) in which another screen is displayed in one screen. ) It has become natural to have functions such as display.
Also in these cases, it is necessary to adjust the black level on each screen. For example, a method of adjusting the black level between the multiple display screens is known (see, for example, Patent Document 1).

In the case of the above-described aspect ratio conversion or multi-screen display such as multi-screen display or P in P display, it is assumed that the image after processing the rectangular image remains rectangular. Yes.
For example, in the technique described in Patent Document 1, a control signal for blanking the display period of the video area is generated, and the video area is out of the black level correction range using this control signal. Processing is performed to keep the black level constant during the display period other than. According to the description in Patent Document 1, it is assumed that the processed image is also rectangular.

  Assuming that the video area of the input image is rectangular, the video area and the peripheral area can be identified by counting the number of pixel data of the input video signal in two directions, horizontal and vertical. Accordingly, the black level adjustment can be performed only on the peripheral area, assuming that the count value of the pixel data count is within a certain range as the peripheral area. By using this technique, black level adjustment of a plurality of screens that do not require complicated circuits can be easily realized.

By the way, there is a case where a non-rectangular video area is included in an image to be subjected to black level adjustment. Hereinafter, such an image is referred to as a “non-rectangular image”.
For example, some image projection apparatuses called so-called projectors include a keystone distortion correction function that corrects an image that is distorted on the screen when the image is projected obliquely on the screen. In this case, the keystone distortion correction image generated on the LCD panel has a video area that is distorted on the contrary to the distortion on the screen. In other words, the keystone distortion corrected image on the LCD panel is a non-rectangular image intentionally generated from a non-rectangular video area. In addition to keystone distortion correction, when an image is projected on a non-planar screen, the image on the LCD panel must be converted into a non-rectangular image in accordance with the shape of the screen surface.

  In the above-described Patent Document 1, black level adjustment for such a non-rectangular image is not assumed. Also, a simple black level adjustment method for a non-rectangular image has not yet been proposed.

  It is also possible to adjust the black level by applying to the non-rectangular image the above-described method of discriminating the video area and its peripheral area based on the number of pixel data. However, since a huge count value corresponding to the entire screen area must be held in advance in a table or the like for each distorted image, in other words, for each angle at the time of oblique projection, such is not practical.

Incidentally, it is also possible to perform keystone distortion correction by first performing signal processing such as gain adjustment. In this case, if the black level adjustment is performed after the signal processing and before the keystone distortion correction, the black level in the peripheral region does not change greatly before and after the correction in the keystone distortion correction, so that the above-described black floating phenomenon can be prevented.
However, if the signal processing is performed before the keystone distortion correction, the keystone distortion correction must be performed in a state where the number of luminance gradations of the pixel data is increased by the signal processing. For example, in the case of an RGB composite digital signal, it is common to increase the number of bits of the pixel data after signal processing by a few bits for each RGB by a few bits because the gradation changes due to signal processing, for each RGB. . For this reason, if the signal processing is performed prior to the keystone distortion correction, there is an adverse effect that the scale of the circuit for performing the keystone distortion correction on the pixel data is increased in a state where the number of bits is increased.
JP-A-6-189214

  The problem to be solved is that there is no simple method for setting the peripheral area other than the video area to a constant level such as a black level for an image including a non-rectangular video area. The black level is affected by the signal processing and fluctuates.

The image processing apparatus of the first aspect of the present invention, deformation processing unit for an input image including a rectangular image area to deform processed image containing the non-rectangular image regions, deforming the shape of Film image regions to a non-rectangular An identification processing unit that identifies whether each pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal; and pixel processing is performed on the transformed image. A signal processing unit to be applied for each of the images, and data for each pixel output from the signal processing unit are input, and based on a control signal from the identification processing unit, the input pixel data is an image of the image subjected to the deformation processing When the data is within the area, the input pixel data is allowed to pass through.When the data is not within the video area of the transformed image, the input pixel data is replaced with data of a certain level and output. Having a replacement control section repeats the control of the replacement with the passage for each pixel data. The identification processing unit includes a level difference setting unit that adds a certain level difference to all pixel data of the input image including the rectangular video region before the transformation process, and a periphery of the rectangular video region before the transformation process. A peripheral level setting unit that replaces a level of pixel data in a region with a constant level smaller than the level difference, and input the pixel data after the transformation process, and determines whether the level of the pixel data is greater than or less than the level difference, A level determination unit that outputs the determination result as the control signal; and a level difference removal unit that removes the level difference from the pixel data after level determination and sequentially outputs the level difference to the signal processing unit .

The image processing apparatus of the second aspect of the present invention, deformation processing unit for an input image including a rectangular image area to deform processed image containing the non-rectangular image regions, deforming the shape of Film image regions to a non-rectangular An identification processing unit that identifies whether each piece of pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal; and for each pixel output from the transformation processing unit When data is input and based on a control signal from the identification processing unit, when the input pixel data is data in a video area of the transformed image, a signal level change is applied to the input pixel data. outputs by performing predetermined signal processing with, and a signal processing section for output without performing the signal processing to the pixel data the input when the pixel data the input is not data of the video area To. The identification processing unit includes a level difference setting unit that adds a certain level difference to all pixel data of the input image including the rectangular video region before the transformation process, and a periphery of the rectangular video region before the transformation process. A peripheral level setting unit that replaces a level of pixel data in a region with a constant level smaller than the level difference, and input the pixel data after the transformation process, and determines whether the level of the pixel data is greater than or less than the level difference, A level determination unit that outputs the determination result as the control signal; and a level difference removal unit that removes the level difference from the pixel data after level determination and sequentially outputs the level difference to the signal processing unit .

In each of the image processing apparatus of the first and second aspects, preferably, pre-Symbol identification processing unit inputs the monotone image with a uniform level in the same rectangular shape as the deformation processing prior to image area, inputs The monotone image is subjected to the same deformation processing as the deformation processing unit, and the pixel data group constituting the monotone image after the deformation processing is used as the control signal for the replacement control unit (in the case of the first aspect) or the signal processing. Part (in the case of the second viewpoint) is sequentially output.

An image processing method according to a first aspect of the present invention includes a deformation processing step of deforming an input image including a rectangular video area into an image including a non-rectangular video area, and deforming the shape of the video area into a non-rectangular shape. An identification processing step of identifying whether or not each pixel data of the transformed image belongs to a video area, and outputting a result of the identification as a control signal; and performing signal processing on the transformed image for each pixel When the signal processing step to be applied to and data for each pixel of the signal-processed image are input, and based on the control signal, the input pixel data is data in the video region of the image subjected to the deformation process Passes the input pixel data, and if it is not data in the video area of the transformed image, replaces the input pixel data with a certain level of data and outputs it. Comprising a replacement control step of repeating control of replacement and pass each pixel data. The identification processing step further includes a level difference setting step of adding a certain level difference to all pixel data of the input image including the rectangular video area before the deformation process, and a rectangular video area before the deformation process. Peripheral level setting step for replacing the pixel data level in the peripheral area with a constant level smaller than the level difference and the pixel data after the deformation process are input, and it is determined whether the level of the pixel data is greater than or less than the level difference and a level determination step of outputting a result of the determination as the control signal, and a level difference removal step of removing the level difference from the after-level determination pixel data, the including.

An image processing method according to a second aspect of the present invention includes a deformation processing step of deforming an input image including a rectangular video region into an image including a non-rectangular video region, and deforming the shape of the video region into a non-rectangular shape. An identification processing step of identifying whether each pixel data of the transformed image belongs to a video area and outputting the identification result as a control signal; and inputting the data for each transformed pixel Based on the control signal, when the inputted pixel data is data in the video region of the transformed image, the inputted pixel data is subjected to predetermined signal processing with a change in signal level and output. And a signal processing step of outputting the input pixel data without performing the signal processing when the input pixel data is not data in the video area . The identification processing step further includes a level difference setting step of adding a certain level difference to all pixel data of the input image including the rectangular video area before the deformation process, and a rectangular video area before the deformation process. Peripheral level setting step for replacing the pixel data level in the peripheral area with a constant level smaller than the level difference and the pixel data after the deformation process are input, and it is determined whether the level of the pixel data is greater than or less than the level difference and a level determination step of outputting a result of the determination as the control signal, and a level difference removal step of removing the level difference from the after-level determination pixel data, the including.

The image processing method of the first and second aspects, preferably, the pre-Symbol identification processing step, type a monotone image with a uniform level in the same rectangular shape as the deformation processing prior to image area, enter monotone The image is subjected to the same deformation process as the step of the deformation process, and pixel data groups constituting the monotone image after the deformation process are sequentially output as the control signal.

An image projection apparatus according to a first aspect of the present invention includes an image display unit and an image displayed on the image display unit when the image is projected obliquely to an external projection surface using light. An image processing unit that deforms an image before projection so that a distorted image generated in the image is corrected, performs predetermined signal processing with signal level conversion on the image after the deformation processing for each pixel, and outputs the processed signal to the image display unit; , to have a. The image processing unit identifies a deformation processing unit that deforms the shape of the video area into a non-rectangular shape by the deformation process, and whether or not each pixel data of the image subjected to the deformation process belongs to the video area. An identification processing unit that outputs a result as a control signal, a signal processing unit that performs signal processing on the transformed image for each pixel, and data for each pixel that is output from the signal processing unit is input, and the identification processing When the input pixel data is data in the video region of the transformed image based on the control signal from the unit, the input pixel data is passed and When it is not data, it has a replacement control section that outputs the input pixel data by replacing it with data of a certain level and repeats the passing and replacement control for each pixel data . The identification processing unit includes a level difference setting unit that adds a certain level difference to all pixel data of the input image including the rectangular video region before the transformation process, and a periphery of the rectangular video region before the transformation process. A peripheral level setting unit that replaces a level of pixel data in a region with a constant level smaller than the level difference, and input the pixel data after the transformation process, and determines whether the level of the pixel data is greater than or less than the level difference, A level determination unit that outputs the determination result as the control signal; and a level difference removal unit that removes the level difference from the pixel data after level determination and sequentially outputs the level difference to the signal processing unit .

An image projection apparatus according to a second aspect of the present invention includes an image display unit and an image displayed on the image display unit when the image is projected obliquely to an external projection surface using light. An image processing unit that deforms an image before projection so that a distorted image generated in the image is corrected, performs predetermined signal processing with signal level conversion on the image after the deformation processing for each pixel, and outputs the processed signal to the image display unit; , to have a. The image processing unit identifies a deformation processing unit that deforms the shape of the video area into a non-rectangular shape by the deformation process, and whether or not each pixel data of the image subjected to the deformation process belongs to the video area. An identification processing unit that outputs a result as a control signal and data for each pixel output from the deformation processing unit are input, and the input pixel data is subjected to the deformation processing based on a control signal from the identification processing unit. If the input pixel data is not data in the video area, the input pixel data is subjected to predetermined signal processing with a change in signal level. a signal processing section for output without the pixel data subjected to the signal processing to have a. The identification processing unit includes a level difference setting unit that adds a certain level difference to all pixel data of the input image including the rectangular video region before the transformation process, and a periphery of the rectangular video region before the transformation process. A peripheral level setting unit that replaces a level of pixel data in a region with a constant level smaller than the level difference, and input the pixel data after the transformation process, and determines whether the level of the pixel data is greater than or less than the level difference, A level determination unit that outputs the determination result as the control signal; and a level difference removal unit that removes the level difference from the pixel data after level determination and sequentially outputs the level difference to the signal processing unit .

According to the first aspect of the present invention, a deformation processing unit (or a deformation processing step) that deforms the shape of the video area of the image indicated by the input signal into a non-rectangular shape, and a signal level change in the image after the deformation processing. A signal processing unit (or signal processing step) that performs accompanying signal processing, and a replacement control unit (or replacement control) that replaces pixel data with data of a certain level only in regions other than the video region of the image after signal processing. Step).
According to the second aspect of the present invention, instead of the replacement control unit in the first aspect, the signal processing unit has a function of performing signal processing only on the pixel data in the video area.
For such replacement control (first viewpoint) or signal processing (second viewpoint), it is determined whether the input pixel data is data in the video area or data outside the video area. There is a need to. This determination is performed based on the control signal, and the control signal is output from the identification processing unit (or identification processing step).

For example, the following two methods are suitable for the identification processing in the present invention.
The first method is to add a level difference to the video area and identify the area depending on whether the signal level is greater than or less than the level difference. More specifically, taking the image processing apparatus as an example, the identification processing unit includes a level difference setting unit, a peripheral level setting unit, a level determination unit, and a level difference removal unit. The level difference setting unit adds a certain level difference to all the pixel data of the input image including the rectangular video area before the deformation process. The peripheral level setting unit replaces the level of the pixel data in the peripheral area of the rectangular video area before the deformation process with a constant level smaller than the level difference. As a result, in the video area, all pixel data is equal to or greater than the level difference, and all of the pixel data in the surrounding area is at a constant level smaller than the level difference. The level discriminating unit discriminates whether the level of the input pixel data is greater than or less than the level difference, and outputs the result of the discrimination as a control signal. The video area with the level difference needs to be returned to the original video level, and therefore, the level difference removal unit performs a process of removing the level difference from the pixel data after the level determination.
In the first aspect, the pixel data after removal of the level difference is sequentially signal-processed by the signal processing unit and then sent to the replacement control unit. In the replacement control unit, the pixel data in the video area is passed as it is, and the pixel data outside the video area is replaced with pixel data of a certain level again. As a result, pixel data outside the video area whose level has been lowered more than necessary by level difference removal is returned to a desired constant level.
In the second aspect, the pixel data after the level difference removal is input to the signal processing unit. The signal processing unit performs signal processing on the pixel data in the video area based on the control signal, and outputs the pixel data outside the video area without performing signal processing.

The second method is a method in which a shape change due to deformation processing is performed on a monotone image, and each pixel data of the monotone image after the shape conversion is used as a control signal. In other words, in the identification processing unit (or identification processing step), a monotone image having the same rectangular shape as the video area before the deformation process is input at a uniform level, and the input monotone image is subjected to the same deformation process as the deformation processing unit. The pixel data group constituting the monotone image after the deformation processing is sequentially output as a control signal to the replacement control unit (first viewpoint) or the signal processing unit (second viewpoint).
In the first aspect, assuming that the monotone image is composed of “1” indicating the video region and “0” other than that, the replacement control unit detects whether the 1-bit control signal is “1” or “0”. In the case of “1”, the input pixel data is passed as it is, and in the case of “0”, the input pixel data is replaced with pixel data of a predetermined constant level.
In the second aspect, the signal processing unit detects whether the 1-bit control signal is “1” or “0”, and in the case of “1”, the input pixel data is output after being subjected to signal processing, In the case of “0”, the input pixel data is output as it is without being subjected to signal processing.

  In the image projection apparatus according to the present invention, for example, an image with a certain level aligned outside the video area by the first or second method described above is displayed on the image display unit and projected onto an external projection surface, for example, a screen. The As a result, a high-quality projected image in which the periphery of the video area is neatly aligned to, for example, the black level appears on the screen.

  The image processing apparatus and method of the present invention, and the image projection apparatus can perform signal processing with a change in signal level on an image that includes a non-rectangular video area by deformation processing. It is easy to align the outside of the video area to a certain level. This constant level replacement process is a process for each pixel data based on the control signal, and the control signal can be easily generated. Therefore, according to the present invention, there is a benefit that a high-quality and stable image can be easily obtained without significantly increasing the circuit scale or the like. Further, since the deformation process can be performed on the pixel data having a bit number smaller than the bit number of the pixel data after the signal processing, it is possible to prevent the configuration of the deformation processing unit from being complicated.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an image projection apparatus (projector) incorporating an image processing apparatus of the present invention and an image conversion method used therefor will be described below with reference to the drawings.

FIG. 1 shows a basic configuration of the projector.
The projector 1 includes an image processing unit 2 including a circuit that performs transformation processing such as keystone distortion correction on a video signal (input signal) and a circuit that performs various signal processing, and a signal obtained by performing transformation processing and various signal processing on an input signal. Has an image display unit 3 for displaying an image (display image) 3a indicated by, for example, an LCD panel. The image processing unit 2 is composed of, for example, one or a plurality of ICs, and constitutes one embodiment of the “image processing apparatus” in the present invention. Note that the image display unit 3 may be configured by an image display device of fixed pixels made of, for example, DMD, other than the LCD.
The projector 1 further includes a light projecting unit 4 including a light source for projecting the display image 3a to the outside, and an optical unit 5 including various lenses. The image display unit 3 may be either a transmission type or a reflection type, but in any case, the display image 3a only needs to be displayed as a projection image 101a on the screen 101 as a projection surface through the optical unit 5. Hereinafter, a case where an LCD panel as a specific example is used as the image display unit 3 will be described.

  The projector 1 includes a relative relationship acquisition unit 6 that acquires relative relationship information indicating a relative relationship between the display image of the LCD panel 3 and the screen 101. The relative relationship acquisition unit 6 is an input unit for inputting relative relationship information from the outside, an external operation means (such as a button), a storage unit (for example, ROM or RAM) that stores presumed relative relationship information, or a relative relationship. There are various forms such as a means for self-detection. The relative relationship acquisition unit 6 acquires, for example, at least the distance of the display image to the screen 101 and the angle formed by the optical axis of the optical unit 5 and the screen surface.

FIG. 2A shows an image projection image when an image is projected obliquely from the lower left position toward the upper right side toward a substantially vertical projection surface (here, a screen). FIG. 2B shows an input image image, and FIG. 2C shows an image image on the LCD panel surface built in the projector.
As shown in FIG. 2A, the projector 1 is arranged and projected at the lower left position toward the screen 101, and the image on the screen 101 looks the same as when projected from the front. Originally, the projected screen should be deformed (keystone deformation) so that the entire screen including the shaded portion in the figure is distorted.
In order to project an image obliquely from the projector 1 having such an arrangement and project it on the screen 101 as if it was projected from the front, it is calculated in advance how the image is distorted depending on the projection position of the projector 1. I have to keep it. At this time, an image that is distorted in the opposite direction to the shape of the image that is distorted when projected obliquely is created, and by projecting it, the image is projected from the front even if projected obliquely to the screen. It can look like a rectangle as it was projected. In order to obtain the projection image as shown in FIG. 2A in the above specific example, the deformation processing unit in the image processing unit 2 shown in FIG. In this manner, the display panel is intentionally deformed and displayed, and this display image is projected onto the screen 101.

  2A is oblique projection, that is, even if the projection is from the upper left, upper right, lower right, or any of the upper, lower, left, and right positions toward the screen 101, the keystone distortion depends on the position. Correction can be performed. The relative positional relationship between the screen 101 and the projector 1 is determined based on the information from the relative information acquisition unit 6 shown in FIG. 1, and the keystone distortion is performed so that a rectangular video area is obtained according to the positional relationship. Correction is applied.

  The image processing unit 2 of the projector 1 further has a function of performing various signal processes in which the signal level changes over the entire image without changing the shape. Examples of such signal processing include gain adjustment, bias adjustment, and gamma correction. Here, the gain adjustment is a process of amplifying the input signal level by multiplying it by a predetermined coefficient and changing the signal amplitude to an appropriate level for LCD driving. The bias adjustment is a process of adding a certain positive or negative offset to the signal amplitude for the purpose of so-called white balance. The gamma correction is a non-linear gain adjustment process that adapts the input signal to the luminance characteristic unique to the display device.

In the embodiment of the present invention, in order to minimize the number of bits of pixel data handled by the keystone distortion correction, these signal processes are performed after the keystone distortion correction. Since the pixel data after the signal processing has a finer gradation expression, the number of bits of each pixel data is set larger than that before the processing. Therefore, if the keystone distortion correction is performed after the signal processing, the scale of the correction circuit increases. In order to avoid such inconvenience, in this embodiment, the keystone distortion correction is performed prior to the signal processing.
The embodiment of the present invention is characterized by processing for preventing the signal level of the peripheral region outside the video region of the image to be processed from changing before and after the signal processing when the keystone distortion correction and the signal processing are sequentially performed. Have Since it is usually desirable to make the peripheral area of the video uniform to the black level (also referred to as “0” level), the process of setting the peripheral area to a certain level is hereinafter referred to as “black level adjustment (processing)”. And

  Hereinafter, a more detailed configuration and operation of the image processing unit capable of adjusting the black level will be described with reference to the drawings.

[First Embodiment]
FIG. 3A shows the configuration of the image processing unit. 3B to 3J schematically show the screens after each processing step. In the drawings and the following description, a screen on which a video area of an image is displayed is referred to as an “effective screen”. Therefore, the “effective screen” here is synonymous with the “video area” in the present invention.
The image processing unit 2 shown in FIG. 3A roughly includes an identification processing unit 2A, a deformation processing unit 2B, a signal processing unit 2C, and a replacement control unit 2D. The processing steps of each part correspond to an embodiment of “identification processing step”, “deformation processing step”, “signal processing step”, and “replacement processing step” in order in the image processing method according to the present invention.

The identification processing unit 2A is a circuit block for identifying a video area of an image, that is, an effective screen and a peripheral area around it.
The identification processing unit 2A in this example includes a level difference setting unit 20, an effective range setting unit 21, a selector 22, a level determination unit 23, and a level difference removal unit 24. Among these, the effective range setting unit 21 and the selector 22 constitute one embodiment of the “peripheral level setting unit” in the present invention.

A video signal is input to the level difference setting unit 20. The video signal may be in any format, such as a signal for each RGB, a luminance signal or a color difference signal, or other combined signals. The level difference setting unit 20 adds (adds) a predetermined level difference, for example, “+1” to the data of the entire screen of the screen (input screen) indicated by the input video signal. As a result, the level difference setting unit 20 outputs one or more image data (FIG. 3B) for the entire screen including the effective screen and the surrounding area.
The effective range setting unit 21 includes a horizontal counter that counts the number of pixels in the horizontal direction and a vertical counter that counts the number of pixels in the vertical direction. The effective range setting unit 21 operates each of the two counters, and a signal indicating an effective screen range having the same rectangular shape as the screen input to the level difference setting unit 20 from the input horizontal synchronization signal and vertical synchronization signal (FIG. 3). (C)) is generated. This signal is input to the selection control terminal of the selector 22.
The output of the level difference setting unit 20 is input to the input 1 of the selector 22, and the black level, that is, “0” level data is input to the input 2 of the selector 22. The selector 22 validates the input 1 side when the input pixel data belongs to the effective screen based on the signal input from the effective range setting unit 21 to the control terminal, and the input pixel data does not belong to the effective screen. Makes the input 2 side valid. For this reason, the output image of the selector 22 shown in FIG. 3D is composed of a rectangular effective screen in which a level of “1 or higher” is guaranteed and a peripheral region of “0” level around it, and the black level is desired. An image adjusted to the value of is output.

The image after the black level adjustment is input to the next image deforming unit 2B and converted into a keystone distortion corrected image (FIG. 3E). FIG. 3E shows, as an example, a keystone distortion correction image when projected from a position near the lower front of the screen. This keystone distorted image is an image distorted in reverse to correct an image that expands upward on the screen when a rectangular image is projected, and has a trapezoidal effective screen with a narrower horizontal width toward the upper side. ing. In the image deformation at the time of correcting the keystone distortion, the original level is almost maintained in the peripheral area of the effective screen, and therefore the “0” level is maintained here.
The keystone distortion correction method is not limited, and in the case of such vertical launch projection, distortion correction can be performed by interpolation processing for each line. In addition, in the case of oblique projection with angles of horizontal and vertical, there are various methods such as a method of performing horizontal and vertical interpolation processing independently or simultaneously, or a method of obtaining a pixel address in advance by calculation using a conversion formula. A distortion correction method can be employed.

The deformed image is input to the level determination unit 23 and the level difference removal unit 24.
In the level discriminating unit 23, the level of each pixel data is compared with a predetermined reference, here “1”, and the result is output as the control signal SC. When the pixel data is “1 or more”, the pixel data is determined to belong to the effective screen, and the logical value of the control signal SC is set to “1”, for example. When the pixel data is “less than 1”, the pixel data Is determined not to belong to the effective screen, and the logical value of the control signal SC is set to “0”, for example. FIG. 3F shows a screen area corresponding to the control signal SC having the logical value “1”. The control signal SC is sequentially input to the replacement control unit 2D.

  In the level difference removing unit 24, “−1” is added to all the pixel data constituting one screen, and the level difference previously set by the level difference setting unit 20 is removed. As a result, pixel data groups each having a desired level are generated on the effective screen corresponding to the original input screen. Further, although the pixel data level is reduced to approximately “−1” in the peripheral area, in this processing, “less than 0” clips the level at “0”, so the black level is maintained in the peripheral area. As a result, even when the black level slightly fluctuates due to noise or the like, the black level adjustment is performed again by this processing.

  The image after the level difference removal is input to the signal processing unit 2C, and first, the gain and bias (G / B) adjustment unit 25 adjusts the gain and bias. In gain adjustment, all pixel data constituting one screen is multiplied by a predetermined coefficient, and in bias adjustment, a predetermined positive or negative bias value is added to all pixel data. Next, the gamma (γ) correction unit 26 multiplies all the pixel data constituting one screen by a coefficient that changes nonlinearly according to the level (luminance value). These signal processes are executed for each pixel data. However, as shown in FIGS. 3H and 3I, the black level (“0” level) is not necessarily guaranteed in the peripheral region after the signal processing.

  The pixel data after the signal processing is sequentially sent to the replacement control unit 2D, and the replacement processing is performed based on the input control signal SC. The replacement process is a process for each pixel data. When the pixel data to be processed belongs to the effective screen with reference to the control signal SC, that is, when the logical value of the control signal SC is “1”, the replacement control unit 2D passes the input pixel data to be processed as it is. Let On the other hand, when the input pixel data to be processed does not belong to the effective screen, that is, when the logical value of the control signal SC is “0”, the replacement control unit 2D converts the input pixel data to be processed into “0” data. Replace with and output.

  The image output from the replacement control unit 2D through the above image processing is a high-quality image in which the peripheral area of the effective screen maintains the black level even if it is a deformed image after signal processing (FIG. 3J). ). This image is displayed on the image display unit (LCD panel) 3 shown in FIG. As a result, the screen 101 has a rectangular image area whose distortion has been corrected, a uniform black level is maintained in the peripheral area, and an easy-to-see high-quality projected image 101a in which so-called black floating is effectively prevented is displayed. Is done.

[Second Embodiment]
FIG. 4A shows the configuration of the image processing unit 2 according to the second embodiment.
In the image processing unit 2 shown in FIG. 4A, the replacement control unit 2D is omitted, and the control signal SC from the level determination unit 23 is a signal processing unit 2C, that is, the gain / bias adjustment unit 25 and the gamma correction unit 26. Are different from the case of the first embodiment shown in FIG. Each of the gain / bias adjustment unit 25 and the gamma correction unit 26 of this example includes a control terminal for enabling processing, and the processing is executed only when the signal input to the control terminal is “1”. It has a function. Since other structures and operations are the same as those in FIG. 3A, description thereof is omitted here.

  The control signal SC obtained by the discrimination process is input from the level discrimination unit 23 to the gain / bias (G / B) adjustment unit 25 and the gamma (γ) correction unit 26, and the image subjected to the image deformation and the discrimination process is a signal. This is input to the gain / bias adjustment unit 25 of the processing unit 2C. When the pixel data to be processed belongs to the effective screen with reference to the input control signal SC, the gain / bias adjustment unit 25 performs gain adjustment and bias adjustment on the pixel data and outputs the pixel data. On the other hand, when the input pixel data does not belong to the valid screen, the gain / bias adjustment unit 25 invalidates the processing and outputs the pixel data without performing processing (gain adjustment and bias adjustment). Similarly, the gamma correction unit 26 refers to the input control signal SC and, when the pixel data to be processed belongs to the effective screen, performs gamma correction on the pixel data and outputs the pixel data. On the other hand, if the input pixel data does not belong to the valid screen, the gamma correction unit 25 invalidates the process and outputs the pixel data without performing the process (gamma correction).

  The image output from the signal processing unit 2C through such processing is an image in which the black level adjusted by the level difference removing unit 24 is maintained, although the signal processing is performed on the effective screen. This image is displayed on the image display unit (LCD panel) 3 shown in FIG. As a result, the screen 101 has a rectangular image area whose distortion has been corrected, a uniform black level is maintained in the peripheral area, and an easy-to-see high-quality projected image 101a in which so-called black floating is effectively prevented is displayed. Is done.

[Third Embodiment]
In this embodiment, another configuration example is shown in which the image processing unit in the first and second embodiments mainly performs identification processing and image deformation processing collectively.
FIG. 5A and FIG. 5B show the configuration of the image processing unit in the third embodiment. Here, FIG. 5A is common to FIG. 3A in which the part after the signal processing shows the first embodiment, and FIG. 5B is the second embodiment in the part after the signal processing. This is common with FIG.

Each of these two image processing units has a deformation processing unit 2E. Each of the deformation processing units 2E functionally includes, for example, a first image deformation unit 30 that inputs an 8-bit R signal, a G signal, and a B signal and performs image deformation processing at the time of keystone distortion correction for each color. And a second image deformation unit 31 that inputs a 1-bit effective screen signal S0 and performs the same image deformation process as each color signal. The second image transformation unit 31 has the function of the “identification processing unit” in the present invention, and outputs a control signal SC indicating a monotone image after image transformation. The control signal SC is a signal indicating a logical value of “1” in the non-rectangular effective screen of the monotone image after image transformation and indicating a logical value of “0” in the peripheral area of the effective screen. Therefore, the same control signal as the control signal output from the level determination unit 23 in the first and second embodiments is output from the deformation processing unit 2E in the present embodiment. The use of this control signal is the same as in the two embodiments described above. In FIG. 5A, the replacement control unit 2D is controlled based on the control signal SC, and in FIG. 5B, the control signal SC is used. To control the signal processing unit 2C. Since details of these controls are repeated, a description thereof is omitted here.
Since the format of the video signal is not limited to the RGB signal, the deformation processing unit 2E can be configured to support signals of other formats.

  Since it depends on the distortion correction method adopted by the deformation processing unit, the comparison of circuit scales cannot be generally described. However, in this embodiment, since the function of the identification processing is provided to the deformation processing unit, at least the identification processing unit The configuration has been simplified.

  As another modified example of the present embodiment, it is possible to generate a pseudo control signal by a method other than the above-described method of performing image deformation at the time of keystone distortion correction on the 1-bit effective screen signal S0. For example, the horizontal and vertical interpolation filters constituting the keystone distortion correction unit 30 for RGB signals usually input several pieces of pixel data and generate one pixel data by interpolation processing. The input pixel data of this interpolation filter The level is monitored every time, and a logic value of “0” is indicated when a predetermined number of pixel data is followed by a “0” level, and a logic value of “1” is indicated otherwise. It may be configured to output the signal SC. In this case, the black level is adjusted not only in the peripheral area but also in places where the black level continues in the effective screen, and is always set to “0” level. However, such a portion of the effective screen is originally used for gain adjustment and gamma correction. After that, it becomes “0” level, and there is a possibility of taking a level different from “0” level only in the case of bias adjustment. However, since the difference is usually very small, the fidelity reproducibility of the image quality of the effective screen is often hardly lowered even by such a method. In this modification, since the peripheral area of the effective screen is reliably adjusted to the black level, the overall quality is improved. In addition, the configuration of the image deformation unit that performs the keystone distortion correction can be simplified as compared with the case of FIGS. 5A and 5B, and the advantage that the circuit scale can be further reduced can be obtained.

In the first to third embodiments, the projection type image display apparatus (image projection apparatus) has been described as an example. However, the image processing apparatus and the image processing method of the present invention can be applied to a direct view type image display apparatus. . Further, the present invention can be applied not only to the keystone correction at the time of oblique projection but also to the case of performing distortion correction when an image is projected on a non-planar screen such as a curved surface.
Furthermore, although each process of the image processing unit 2 according to the first to third embodiments of the present invention is assumed to be realized in a form including hardware (electronic circuit), a part of the process or It is also possible to implement the whole on software stored in advance in a microcomputer, a central processing unit (CPU) or the like.

  INDUSTRIAL APPLICABILITY The present invention can be suitably applied to an image processing apparatus, for example, an image processing IC, with a function of generating an image including a non-rectangular video area built in a direct view or projection type image display apparatus. The present invention can be suitably applied to an image projection apparatus having a function of correcting image distortion during oblique projection by a projector or the like.

It is a figure which shows the basic composition of the projector concerning embodiment of this invention with the screen which is the projection surface of the image. (A) is a figure which shows the image projection image at the time of projecting an image to diagonally upper right from the position of the lower left toward a substantially vertical screen. (B) is a diagram showing an image of an input image, and (C) is a diagram showing an image image on the LCD panel surface. (A) is a figure showing composition of an image processing part concerning a 1st embodiment of the present invention. (B)-(J) are figures which show the screen after each processing step typically. (A) is a figure which shows the structure of the image process part concerning the 2nd Embodiment of this invention. (B)-(I) is a figure which shows typically the screen after each process step. (A) And (B) is a figure which shows the structure of the image process part concerning the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Projector (image projection apparatus), 2 ... Image processing part, 2A ... Identification processing part, 2B, 2E ... Image deformation part, 2C ... Signal processing part, 2D ... Replacement control part, 3 ... LCD panel (image display part) 3 ... Display image, 4 ... Projection unit, 5 ... Optical unit, 6 ... Relative relationship acquisition ... Level difference setting unit, 21 ... Selector (peripheral level setting unit), 22 ... Effective range setting unit (peripheral level setting unit) , 23... Level discriminating unit, 24... Level difference removing unit, 25... Gain / bias adjusting unit, 26... Gamma correcting unit, 30 ... first image deforming unit, 31 ... second image deforming unit (identification processing unit) 101 ... Screen (projection surface), 101a ... Projected image, SC ... Control signal, S ... Signal indicating monotone image

Claims (10)

And deformation processing unit for an input image including a rectangular image area to deform processed image containing the non-rectangular image regions, deforming the shape of Film image regions to a non-rectangular,
An identification processing unit that identifies whether or not each pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal;
A signal processing unit that applies signal processing to the transformed image for each pixel;
When data for each pixel output from the signal processing unit is input, and based on a control signal from the identification processing unit, the input pixel data is data in a video area of the image subjected to the deformation processing When the input pixel data is passed and the data is not in the image area of the transformed image, the input pixel data is replaced with a certain level of data and output, and the pass and replacement control is performed for each pixel data. A replacement control unit that repeats;
Have
The identification processing unit
A level difference setting unit that adds a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting unit that replaces a level of pixel data in a peripheral area of the rectangular video area before the transformation process with a constant level smaller than the level difference;
A level determination unit that inputs the pixel data after the deformation process, determines whether the level of the pixel data is greater than or less than the level difference, and outputs a result of the determination as the control signal;
A level difference removing unit that removes the level difference from the pixel data after the level determination and sequentially outputs to the signal processing unit;
An image processing apparatus. The identification processing unit inputs a uniform monotone image in the same rectangular shape as the video area before the deformation process, performs the same deformation process as the deformation processing unit on the input monotone image, and after the deformation process The image processing apparatus according to claim 1, wherein the pixel data group constituting the monotone image is sequentially output to the replacement control unit as the control signal. And deformation processing unit for an input image including a rectangular image area to deform processed image containing the non-rectangular image regions, deforming the shape of Film image regions to a non-rectangular,
An identification processing unit that identifies whether or not each pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal;
When data for each pixel output from the deformation processing unit is input, and based on a control signal from the identification processing unit, the input pixel data is data in a video area of the image subjected to the deformation processing The input pixel data is output after being subjected to predetermined signal processing with a change in signal level, and when the input pixel data is not data in the video area, the input pixel data is output without being subjected to the signal processing. A signal processing unit
Have
The identification processing unit
A level difference setting unit that adds a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting unit that replaces a level of pixel data in a peripheral area of the rectangular video area before the transformation process with a constant level smaller than the level difference;
A level determination unit that inputs the pixel data after the deformation process, determines whether the level of the pixel data is greater than or less than the level difference, and outputs a result of the determination as the control signal;
A level difference removing unit that removes the level difference from the pixel data after the level determination and sequentially outputs to the signal processing unit;
An image processing apparatus. The identification processing unit inputs a uniform monotone image in the same rectangular shape as the video area before the deformation process, performs the same deformation process as the deformation processing unit on the input monotone image, and after the deformation process The image processing apparatus according to claim 3 , wherein pixel data groups constituting a monotone image are sequentially output to the signal processing unit as the control signal. A transformation process step of transforming an input image including a rectangular video area into an image including a non-rectangular video area, and transforming the shape of the video area into a non-rectangular shape;
An identification processing step of identifying whether each pixel data of the transformed image belongs to a video area, and outputting a result of the identification as a control signal;
A signal processing step of applying signal processing to the transformed image for each pixel;
Data for each pixel of the signal-processed image is input, and when the input pixel data is data in the video area of the image subjected to the deformation process based on the control signal, the input pixel data is A replacement control step that passes and replaces the input pixel data with data at a certain level when the data is not within the video region of the transformed image, and repeats the pass and replacement control for each pixel data; and
Including
The identification processing step further includes
A level difference setting step of adding a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting step of replacing the level of the pixel data in the peripheral area of the rectangular video area before the deformation process with a constant level smaller than the level difference;
A level determination step of inputting the pixel data after the deformation process, determining whether the level of the pixel data is greater than or less than the level difference, and outputting the result of the determination as the control signal;
A level difference removing step for removing the level difference from the pixel data after level discrimination;
Including images processing methods. In the identification processing step, a monotone image having the same rectangular shape as that of the video area before the deformation process is input at a uniform level, the input monotone image is subjected to the same deformation process as the deformation process step, and after the deformation process, The image processing method according to claim 5 , wherein pixel data groups constituting the monotone image are sequentially output as the control signal. A transformation process step of transforming an input image including a rectangular video area into an image including a non-rectangular video area, and transforming the shape of the video area into a non-rectangular shape;
An identification processing step of identifying whether each pixel data of the transformed image belongs to a video area, and outputting a result of the identification as a control signal;
Data for each pixel subjected to the deformation process is input, and when the input pixel data is data in a video area of the image subjected to the deformation process based on the control signal, a signal level is set to the input pixel data. A signal processing step of performing a predetermined signal processing with a change of the output, and outputting the input pixel data without performing the signal processing when the input pixel data is not data in the video area; and
Including
The identification processing step further includes
A level difference setting step of adding a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting step of replacing the level of the pixel data in the peripheral area of the rectangular video area before the deformation process with a constant level smaller than the level difference;
A level determination step of inputting the pixel data after the deformation process, determining whether the level of the pixel data is greater than or less than the level difference, and outputting the result of the determination as the control signal;
A level difference removing step for removing the level difference from the pixel data after level discrimination;
Including images processing methods. In the identification processing step, a monotone image having the same rectangular shape as that of the video area before the deformation process is input at a uniform level, the input monotone image is subjected to the same deformation process as the deformation process step, and after the deformation process, The image processing method according to claim 7 , wherein pixel data groups constituting the monotone image are sequentially output as the control signal. An image display unit;
When the image displayed on the image display unit is projected obliquely to an external projection surface using light, the image before projection is transformed so that a distortion image generated on the projection surface is corrected, a prescribed signal processing with an image signal level converted to after the transformation process and the image processing section to be output to the image display unit applied to each pixel, the possess,
The image processing unit
A deformation processing unit that deforms the shape of the video area into a non-rectangular shape by the deformation processing;
An identification processing unit that identifies whether or not each pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal;
A signal processing unit that applies signal processing to the transformed image for each pixel;
When data for each pixel output from the signal processing unit is input, and based on a control signal from the identification processing unit, the input pixel data is data in a video area of the image subjected to the deformation processing When the input pixel data is passed and the data is not in the image area of the transformed image, the input pixel data is replaced with a certain level of data and output, and the pass and replacement control is performed for each pixel data. A replacement control unit that repeats;
Have
The identification processing unit
A level difference setting unit that adds a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting unit that replaces a level of pixel data in a peripheral area of the rectangular video area before the transformation process with a constant level smaller than the level difference;
A level determination unit that inputs the pixel data after the deformation process, determines whether the level of the pixel data is greater than or less than the level difference, and outputs a result of the determination as the control signal;
A level difference removing unit that removes the level difference from the pixel data after the level determination and sequentially outputs to the signal processing unit;
Including an image projection apparatus. An image display unit;
When the image displayed on the image display unit is projected obliquely to an external projection surface using light, the image before projection is transformed so that a distortion image generated on the projection surface is corrected, An image processing unit that performs predetermined signal processing with signal level conversion on the image after deformation processing for each pixel and outputs the image to the image display unit;
I have a,
The image processing unit
A deformation processing unit that deforms the shape of the video area into a non-rectangular shape by the deformation processing;
An identification processing unit that identifies whether or not each pixel data of the transformed image belongs to a video area, and outputs a result of the identification as a control signal;
When data for each pixel output from the deformation processing unit is input, and based on a control signal from the identification processing unit, the input pixel data is data in a video area of the image subjected to the deformation processing The input pixel data is output after being subjected to predetermined signal processing with a change in signal level, and when the input pixel data is not data in the video area, the input pixel data is output without being subjected to the signal processing. A signal processing unit
Have
The identification processing unit
A level difference setting unit for adding a certain level difference to all the pixel data of the input image including the rectangular video area before the transformation process;
A peripheral level setting unit that replaces a level of pixel data in a peripheral area of the rectangular video area before the transformation process with a constant level smaller than the level difference;
A level determination unit that inputs the pixel data after the deformation process, determines whether the level of the pixel data is greater than or less than the level difference, and outputs a result of the determination as the control signal;
A level difference removing unit that removes the level difference from the pixel data after the level determination and sequentially outputs to the signal processing unit;
An image processing apparatus.

Images