JP2020202527A - Image processing device, image display device, and image processing method - Google Patents

Abstract

To perform good deformation processing on an image while reducing the burden on a user.SOLUTION: An image processing device 140 calculates the amount of movement to move a plurality of second points located on a plurality of sides other than a first point from the movement amount given to the plurality of first points located on the plurality of sides of an image, calculates the amount of movement to move a plurality of third points located in an image area inside the plurality of sides from the movement amount of the first and second points. Then, a deformation process is performed according to the movement amount of the first, second, and third points.SELECTED DRAWING: Figure 1

Description

The present invention relates to image processing that adds geometric deformation to an image.

For the purpose of reducing (correcting) distortion of the projected image, the projector is subjected to image transformation processing that deforms the projected image according to the user's instruction of the amount of movement to the four corner points or multiple areas of the projected image. There is something you can do. However, as the number of movement instructions by the user increases, the burden on the user increases. Patent Document 1 discloses a projector capable of changing the number of movement amounts instructed by a user by making it possible to change the number of divisions of a region in a projected image. Further, Patent Document 2 discloses a projector in which a typical deformation pattern is prepared in advance so that a user can select the amount of movement of each side of the projected image in each deformation pattern.

Japanese Unexamined Patent Publication No. 2013-078001 Japanese Unexamined Patent Publication No. 2013-077988

However, in the projector disclosed in Patent Document 1, if the number of region divisions is reduced and the number of movement amounts indicated by the user is reduced, the projected image cannot be finely deformed, and distortion correction and the like cannot be performed satisfactorily. Further, in the projector disclosed in Patent Document 2, since the deformation pattern prepared in advance is limited and only each side can be moved, it may be difficult to deform the projected image as desired by the user. ..

The present invention provides an image processing apparatus or the like capable of performing good deformation processing on an image while reducing the burden on the user.

The image processing apparatus as one aspect of the present invention performs deformation processing for deforming an image having a plurality of sides. The image processing device moves a plurality of second points located on a plurality of sides other than the first point from a movement amount given to a plurality of first points located on the plurality of sides. The side movement amount calculating means for calculating the movement amount to be moved, and the movement amount for moving a plurality of third points located in the image area inside the plurality of sides from the movement amount of the first and second points. It is characterized by having a region movement amount calculation means for calculation and a deformation processing means for performing deformation processing according to the movement amount of the first, second and third points.

An image display device having the image processing device and a display means for displaying the image subjected to the deformation processing in the image processing device also constitutes another aspect of the present invention.

Further, the image processing method as another aspect of the present invention is a method of performing deformation processing for deforming an image having a plurality of sides. In the image processing method, a plurality of second points located on a plurality of sides other than the first point are moved from a movement amount given to the plurality of first points located on the plurality of sides. A step of calculating the movement amount to be moved, and a step of calculating the movement amount of moving a plurality of third points located in the image area inside the plurality of sides from the movement amount of the first and second points. , The first, the second, and the third point are characterized by having a step of performing a deformation process according to the amount of movement.

A computer program that causes a computer to perform processing according to the above image processing method also constitutes another aspect of the present invention.

According to the present invention, it is possible to perform good deformation processing on an image while reducing the burden on the user instructing the movement amount.

The block diagram which shows the structure of the projector which is Example 1 of this invention. The figure which shows the example of the given movement amount. The flowchart which shows the image transformation processing which the projector of Example 1 executes. The figure which shows the interpolation result of the movement amount on each side of a projected image. The figure which shows the interpolation result of the movement amount of the inner image area of a projected image. The graph which shows the interpolation result of FIG. The figure which shows the example of the interpolation result in Example 1.

Hereinafter, examples of the present invention will be described with reference to the drawings.

FIG. 1 shows the configuration of a projector 100 as an image display device according to a first embodiment of the present invention. The projector 100 controls the light transmittance or reflectance of the light modulation element according to the input image signal, and projects the light modulated by the light modulation element onto a projected surface such as a screen to produce a projected image. Is displayed.

The projector 100 includes a CPU 110, a ROM 111, a RAM 112, an EEPROM 114, an operation unit 113, an image input unit 130, and an image processing unit 140. Further, the projector 100 includes a liquid crystal control unit 150, a liquid crystal element 151R, a liquid crystal element 151G, a liquid crystal element 151B, a light source control unit 160, a light source 161 and a color separation unit 162, a color synthesis unit 163, an optical system control unit 170, and a projection optical system 171. Has. Further, the projector 100 has a communication unit 193, a display control unit 195, and a display unit 196.

The CPU 110 controls the operation of each operation block of the projector 100. The ROM 111 stores a control program that describes the processing procedure of the CPU 110. The RAM 112 temporarily stores a control program and data as a work memory. The EEPROM 114 can store various data without requiring power supply. Further, the CPU 110 performs power control processing and OSD processing according to an instruction signal received by the communication unit 193 from an external device (not shown) such as a personal computer or a remote control unit by wired, wireless or infrared communication, and also performs an image processing unit. Let 140 perform image quality adjustment processing, image deformation processing, and the like.

Further, the operation unit 113 is composed of operation members such as switches and dials and a touch panel provided on the display unit 196, and receives a user's operation and outputs an instruction signal to the CPU 110. The CPU 110 performs the various processes described above in response to the instruction signal.

The image input unit 130 receives an image signal (video signal) input from an image supply device (not shown). When an analog image signal is received, the analog image signal is converted into a digital image signal, and when a digital image signal is received, a predetermined decoding process or the like is performed and output to the image processing unit 140. The image supply device is a personal computer, a camera, a smartphone, a hard disk recorder, a game machine, or the like.

The image processing unit 140 as an image processing device is configured by an image processing microprocessor as a computer, and has frame thinning processing, frame interpolation processing, resolution conversion processing, and an image for an image signal input from the image input unit 130. Image processing such as deformation processing is performed to generate image data, which is output to the liquid crystal control unit 150.

The liquid crystal control unit 150 controls the voltage applied to each pixel of the liquid crystal elements 151R, 151G, and 151B as the light modulation element based on the image data generated by the image processing unit 140, thereby producing an original image on each liquid crystal element. Form. As the applied voltage of each pixel changes, the transmittance or reflectance of each pixel changes, and the light transmitted or reflected by each pixel is modulated (the amount of light is changed). The display means is configured by the light modulation element. As the light modulation element, a digital micromirror device may be used instead of the liquid crystal element.

The light source 161 is configured by using a discharge lamp or a laser diode, and emits illumination light which is white light. The light source control unit 160 controls the on / off of the light source 161 and the amount of light emitted. The color separation unit 162 is composed of a dichroic mirror, a polarization beam splitter, and the like, and separates the illumination light into red (R) light, green (G) light, and blue (B) light. The liquid crystal elements 151R, 151G, and 151B modulate the incident R light, G light, and B light, respectively, as described above.

The color synthesis unit 163 is composed of a dichroic mirror, a polarizing beam splitter, and the like, and synthesizes R light, G light, and B light modulated by the liquid crystal elements 151R, 151G, and 151B to generate color image light. The projection optical system 171 projects color image light onto the screen 180. As a result, the projected image corresponding to the input image signal is displayed on the screen 180. The optical system control unit 170 controls operations such as zooming, shifting, and focusing of the projection optical system 171.

The display control unit 195 controls the display unit 196 so that an image such as an operation screen or a switch icon for the user to operate the projector 100 is displayed. The display unit 196 is composed of a liquid crystal display, an organic EL display, or the like. In addition, instead of the display unit 195, an operation button for operating the projector 100 may be used.

Next, an image transformation process for transforming a projected image having a plurality of sides (four in this embodiment) in this embodiment will be described. Note that deforming the projected image means deforming the original image actually formed on the liquid crystal elements 151R, 151G, 151B. In this embodiment, 17 horizontal and 10 vertical pixels (hereinafter referred to as points) are set in two dimensions on the projected image (liquid crystal element), and a movement amount is given for each point. The image deformation process is used to correct geometric distortion such as trapezoidal distortion of a projected image, and to reduce registration deviation by calculating the amount of movement for each point in each of the light modulation elements for RGB. Further, the points may be calculated not for each pixel of the projected image but for each unit (sub-pixel) smaller than the pixel.

The upper and lower figures of FIG. 2 show the amount of vertical movement given by the user or an instruction from an external device to the points located at the four corners and the points located between the upper sides of the projected image (hereinafter referred to as the vertical movement amount). ) And the amount of lateral movement (hereinafter referred to as the amount of lateral movement). The points on the four corners and the upper side are called representative points (first points).

For example, when each liquid crystal element has 3840 × 2160 pixels, the vertical and horizontal movement amounts (first movement amount) shown in the elements 201 in the upper and lower figures in FIG. 2 are coordinates (0,0). Indicates the amount of vertical and horizontal movement of the representative points of. Similarly, the vertical and horizontal movement amounts of the element 202 indicate the vertical and horizontal movement amounts of the representative points of the coordinates (1920, 0), and the vertical and horizontal movement amounts of the element 203 are the vertical and horizontal movement amounts of the representative points of the coordinates (3839, 0). Indicates the amount of lateral movement. The vertical and horizontal movement amount of the element 204 indicates the vertical and horizontal movement amount of the representative point of the coordinates (0,2159), and the vertical and horizontal movement amount of the element 205 is the vertical and horizontal movement amount of the representative point of the coordinates (3839,2159). Is shown.

In this embodiment, the amount of movement given to points other than the representative points (hereinafter referred to as interpolation points) is calculated by linear interpolation processing using the amount of movement of the representative points. The method of calculating the movement amount at this time will be described below. Since the vertical movement amount and the horizontal movement amount are calculated independently of each other and by the same linear interpolation processing, only the calculation method of the vertical movement amount will be described with reference to FIG.

The flowchart of FIG. 3 shows an image deformation process (image processing method) executed by an image processing unit 140 as a side movement amount calculation means, an area movement amount calculation means, and a deformation processing means according to a computer program. In step 300, the image processing unit 140 is given a vertical movement amount to the representative point according to an instruction from the user or the like as shown in FIG.

Next, in steps 301 to 304, the image processing unit 140 calculates the movement amount (second movement amount) of the interpolation points (second points) located on the upper side, the lower side, the right side, and the left side of the interpolation points. Perform interpolation processing. First, in step 301, the image processing unit 140 calculates the amount of movement of the interpolation point on the upper side. Since the interpolation here is linear interpolation of one-dimensional data, the amount of movement of the interpolation point on the upper side is

Explain as given in. The subscript i indicates the horizontal coordinates of the upper side. In this embodiment, the interpolation evaluation value J is set to

Give in. The unknown movement amount that minimizes the interpolation evaluation value J is obtained. The amount of movement of the interpolation point on the upper side obtained by this interpolation processing is shown in FIG. 4A with a gray background.

Next, in step 302, the image processing unit 140 performs the same interpolation processing as in step 301 to calculate the amount of movement of the interpolation point located on the lower side. The calculated movement amount of the interpolation point on the lower side is shown in FIG. 4B with a gray background.

Next, in step 303, the image processing unit 140 calculates the amount of movement of the interpolation point (second point) on the right side. Since the interpolation here is also linear interpolation of one-dimensional data, the amount of movement of the interpolation point on the right side is

Explain as given in. The subscript i indicates the vertical coordinates of the right side. In this embodiment, the interpolation evaluation value J is set to

Give in. An unknown amount of movement is obtained so that the interpolation evaluation value J'is minimized. The amount of movement of the interpolation point on the right side obtained by this interpolation processing is shown in FIG. 4C with a gray background.

Next, in step 304, the image processing unit 140 performs the same interpolation processing as in step 303 to calculate the amount of movement of the interpolation point located on the left side. The calculated movement amount of the interpolation point on the left side is shown in FIG. 4D with a gray background. The processes of steps 301 to 304 may be performed by changing their order or at the same time.

Next, in step 305, the image processing unit 140 linearly aligns the movement amount (third movement amount) of the interpolation point (third point) located in the area inside the four sides (hereinafter referred to as the inner image area). Calculated by interpolation processing. Since the interpolation here is the interpolation of two-dimensional data, the amount of movement of the interpolation point is

Explain as given in. The subscript i indicates the horizontal coordinates, and the subscript j indicates the vertical coordinates. The unknown movement amount is calculated by using the movement amount of the representative point and the interpolation point for which the movement amount was calculated in steps 302 to 304 as the known movement amount. In this embodiment, the interpolation evaluation value J is set to

Give in. The unknown movement amount that minimizes the interpolation evaluation value J is obtained. The amount of movement of the interpolation points in the inner image area obtained by this interpolation processing is shown in FIG. 5A with a gray background. FIG. 6A shows a graph that visualizes the movement amount of the representative points and the movement amount of all the interpolated points. In FIG. 6A, the two horizontal axes indicate the vertical direction and the horizontal direction, and the vertical axis indicates the amount of movement.

On the other hand, in FIGS. 5 (B) and 6 (B), unlike the present embodiment, the movement amounts of all the interpolation points are used as representative points without first calculating the movement amounts of the interpolation points on the four sides. The result calculated by the interpolation process from the movement amount is shown. In FIGS. 5 (B) and 6 (B), the movement amount of the interpolation points on the side unrelated to the side for which the movement amount of the interpolation points is calculated changes, and the movement amounts of all the interpolation points are changed. It is difficult to determine. On the other hand, according to this embodiment, such difficulty can be solved.

FIG. 7A shows the result of interpolation processing when the representative points in this embodiment are different from those in FIG. In FIG. 7 (A), in addition to the representative points on the four corners and the upper side, the representative point (first point) located in the middle on the lower side and the representative point (fourth point) located in the inner image area are The movement amount is given to them according to the instruction of the user or the like. Then, from the movement amount of these representative points, the movement amount of the interpolation points (second and third points) whose background is gray is calculated by the process shown in FIG. FIG. 7B shows a graph that visualizes the movement amounts of all the representative points and the interpolation points as in FIG. 6B.

As described above, according to the present embodiment, it is possible to perform good image transformation processing while reducing the number of movement amounts instructed by the user, that is, the burden on the user.

In this embodiment, the case where the movement amount of the interpolation point on the side of the projection pixel is calculated by linear interpolation has been described, but it may be calculated by projective transformation.
(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Each of the above-described examples is only a representative example, and various modifications and changes can be made to each of the examples in carrying out the present invention.

100 Projector 140 Image processing unit

Claims (10)

An image processing device that performs transformation processing that transforms an image that has multiple sides.
A movement amount for moving a plurality of second points located on the plurality of sides other than the first point from a movement amount given to the plurality of first points located on the plurality of sides. A means for calculating the amount of side movement to calculate
An area movement amount calculating means for calculating a movement amount for moving a plurality of third points located in an image area inside the plurality of sides from the movement amounts of the first and second points.
An image processing apparatus comprising: a deformation processing means for performing the deformation processing according to the amount of movement of the first, second, and third points. The region movement amount calculation means is based on the movement amount of the first and second points and the movement amount given to at least one fourth point located in the image region, and the fourth point. The image processing apparatus according to claim 1, wherein the amount of movement of the third point other than the above is calculated. The image processing apparatus according to claim 1 or 2, wherein the plurality of first points include at least points at four corners of the image. The image processing apparatus according to claim 3, wherein the plurality of first points further include a point located in the middle of the side. The image processing apparatus according to any one of claims 1 to 4, wherein the side movement amount calculation means calculates the movement amount of the second point by linear interpolation or projective transformation. The image processing apparatus according to any one of claims 1 to 5, wherein the area movement amount calculation means calculates the movement amount of the third point by linear interpolation. The image processing apparatus according to any one of claims 1 to 6, wherein each point is a pixel of the image or a unit smaller than the pixel. The image processing apparatus according to any one of claims 1 to 7.
An image display device comprising a display means for displaying an image that has undergone the deformation process in the image processing device. It is an image processing method that performs transformation processing that transforms an image having a plurality of sides.
A movement amount for moving a plurality of second points located on the plurality of sides other than the first point from a movement amount given to the plurality of first points located on the plurality of sides. And the steps to calculate
From the movement amount of the first and second points, a step of calculating the movement amount of moving a plurality of third points located in the image area inside the plurality of sides, and
An image processing method comprising a step of performing the deformation process according to the amount of movement of the first, second, and third points. A computer program comprising causing a computer to perform processing according to an image processing method according to claim 9.