JP6285657B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image transformation operation technique for a display target image by an image processing apparatus. An example of the usage is related to a technique for operating a deformed shape in a function generally known as a keystone correction function (or trapezoid correction function) of a projector.

  In an image processing apparatus, there are many cases where an image transformation process is required for an image to be processed. For example, in a projector product, an image deformation process called a keystone (trapezoid) correction process is performed. More specifically, when the output light of the projector is projected on the screen, the effective area projected on the screen is distorted in a trapezoid shape due to the installation inclination angle of the projector, optical lens shift, and the like. Since it is difficult for the user to see this trapezoidal distortion, the effective area is deformed into an inverted trapezoidal shape, and the image is deformed so that the effective area projected on the screen is rectangular. This image deformation process is generally known as a keystone (trapezoid) correction process.

  The method for the user to operate the keystone correction process is mainly to set the tilt of the projector housing with respect to the screen (optical tilt designation method) and the positions of the four corner points of the image projected on the screen. There is a method of moving and setting (four-point designating deformation method).

  The optical tilt designation method includes an input device for setting the vertical / horizontal tilt of the projector housing, and the projector uses a screen based on the vertical / horizontal tilt information set via the input device. Deformation processing is performed so as to reversely correct the above optical distortion.

  For example, when the projector is set upward with an angle with respect to the screen, the projection surface on the screen has a trapezoidal shape spreading upward. At this time, the user sets the tilt in the vertical direction to the projector by the optical tilt designation method, and the projector deforms in a direction to cancel the distortion of the projection plane caused by the set vertical tilt.

  Specifically, when an upward tilt angle with respect to the screen is input, the projector deforms the input image into a trapezoidal shape that spreads downward. As a result, the projected image on the screen is displayed as a rectangular shape with optical distortion canceled. However, the optical tilt designation method can make the projected image on the screen rectangular, but it cannot be placed at the position intended by the user, such as aligning the rectangular projected image with the screen frame. In order to match the rectangular projection image with the screen frame, operations such as zoom and lens shift are required in addition to the optical tilt designation method.

  On the other hand, the four-point designating method is a method for transforming the projected image by moving the coordinates of the four corners of the projected image on the screen. In addition to correcting the projected image on the screen to a rectangle, the optical tilt is designated. There is an advantage that it can be arranged at any position that cannot be achieved by the method.

  By the way, in the image deformation process, the shape that can be deformed is restricted due to the mounting configuration. For example, as one of the methods for performing image deformation processing, there is a method in which an image is first fetched into a memory, subjected to deformation processing, and then output. In this method, the limit value of the deformable shape varies depending on the resolution of the input image and the capacity of the memory. In general, the higher the resolution, the more memory is used, so the limit value of the shape that can be deformed is more limited. For this reason, a configuration for notifying the user of the limit of the shape that can be deformed even in the four-point designated deformation method is required.

  As an example of this configuration, there is a method disclosed in Patent Document 1. In the method of Patent Document 1, in the four-point designating modification method, marks are displayed in the movable direction of the vertex selected by the user among the four corners of the projected image on the screen. Here, when the vertex selected by the user is located at the limit of the deformation range and cannot move in a predetermined direction, the user is notified of the limit of the shape that can be deformed by making the direction mark corresponding to the direction that cannot be moved inconspicuous. To do.

JP 2010-250041 A

  When the user attempts to deform the projected image by the four-point designated deformation method and place the deformed shape at a predetermined position such as a screen frame, the user determines whether the projected image can be deformed and placed on the screen frame. I need to know.

  However, in the method described in Patent Document 1, it is not possible to determine whether or not the selected vertex can be transformed into the shape intended by the user unless the selected vertex is actually moved to the intended shape. If any of the four vertices reaches the limit of the deformation range and cannot be formed into the shape intended by the user, the user needs to readjust the corresponding vertex and cause rework.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an efficient image deformation operation by presenting a deformable range of a projected image before the deformation.

In order to achieve the above object, an image processing apparatus according to the present invention comprises the following arrangement. That is,
Deformation means for executing deformation processing for generating a deformation image from an input image;
Designating means for designating the coordinates of the input image;
Generating means for generating a range image indicating a movable range based on deformation processing by the deforming means of the coordinates specified by the specifying means based on a positional relationship between the specified coordinates and other coordinates of the input image. When,
Projection control means for projecting the range image generated by the generating means onto the projecting means;
Have

  According to the present invention, the deformable range of the projected image is presented before the deformation, and an efficient image deformation operation can be realized.

It is a figure which shows the function structure of an image processing apparatus. It is a figure which shows a data structure. It is a figure explaining deformation capability information. It is a flowchart which shows the process of a deformable range production | generation part. It is a figure explaining the image produced | generated in a process. It is a figure explaining a deformable range image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Embodiment 1>
FIG. 1 is a diagram illustrating a functional configuration of an image transformation apparatus that is one of the image processing apparatuses according to the first embodiment. 1 has standard components (for example, CPU, RAM, ROM, hard disk, external storage device, network interface, display, keyboard, mouse, etc.) mounted on a general-purpose computer. ing. These constituent elements realize various functional constituent elements of the image deformation apparatus 101 shown in FIG.

  The image deforming apparatus 101 in FIG. 1 superimposes information indicating how much the vertexes of the four corners can be deformed in the deformed shape by deforming the input image 100 to be projected onto the projection surface, and output the output image. 102 is generated. In the following description, in the first embodiment, as points on the input image, which are used as a reference for the user to perform an image deformation instruction, for example, each vertex at the four corners of the image is represented as “representative”. “Point” and the vertex selected by the user are described as “selected representative point”, respectively. The detailed data structure of each of the deformation capability information 113, representative point selection information 114, deformation control information 115, representative point information 116, and deformable range information 117 shown in FIG. 1 is shown in FIG.

  Hereinafter, the configuration of the image deformation device 101 will be described in detail. The input image 100 input to the image deformation device 101 is input to the image deformation unit 104. The image deforming unit 104 receives the input image 100 and the deformation control information 115, reads / writes the image data 111 from / to the memory 103, and outputs the deformed image 112 and the deformability information 113. Here, as shown in FIG. 2, the deformation control information 115 is information composed of four sets of the representative point coordinates before deformation and the corresponding representative point coordinates after deformation, and the relationship between the coordinates before and after the deformation. Is shown. In the first embodiment, since the four vertices of the input image 100 are used as representative points, the representative point coordinates before deformation are the coordinates of the four vertices of the rectangle of the input image 100.

  The coordinates may be coordinates in a screen coordinate system defined on the basis of a predetermined point (origin) on the screen as a projection plane, or are virtually defined in the image transformation apparatus 101. Coordinates in a virtual coordinate system may be used. In any case, it is only necessary to previously define a coordinate system that can be handled uniformly in the image deformation process and to realize the image deformation process according to the coordinate system. This coordinate system means, for example, a two-dimensional coordinate system defined by the X axis and the Y axis. In this embodiment, a screen coordinate system in which the horizontal direction is the X axis and the vertical direction is the Y axis will be described as an example.

  The image transformation unit 104 writes the input image 100 into the memory 103 while modifying the input image 100 according to the deformation designated by the deformation control information 115, reads out the completed line of the deformed image on the memory 103, and outputs the deformed image 112. To do. In addition, the image deformation unit 104 outputs deformation capability information 113 indicating the image deformation capability of the image deformation unit 104, which is necessary for generating a deformable range of each representative point. As the deformability information 113, there are several examples depending on the implementation of the image deforming unit 104. Here, as shown in FIG. 2, an example using maximum inclination information is shown.

  Here, the relationship between the image data 111 written in the memory 103 from the image transformation unit 104 is shown in FIG. As shown in FIG. 3, when the first line of the input image 100 is input to the image transformation unit 104, it corresponds to input of one line of the input image along the address connecting the representative point 502 and the representative point 503. Memory write 501 occurs. For this reason, the memory 103 needs to hold data having a capacity corresponding to the number of lines 504 held in the memory 103 as shown in FIG. Usually, since the capacity of the memory 103 is determined when the image deformation apparatus 101 is designed, the number of lines 504 held in the memory has an upper limit, and the relationship between the representative points must be within this upper limit.

  In the keystone deformation, the inclination of the straight line connecting the representative points in memory writing corresponding to the input of one line of the input image has the maximum inclination on either the upper side or the lower side of the deformed square. . That is, since the gradient of either the straight line connecting the representative point 502 and the representative point 503 or the straight line connecting the representative point 506 and the representative point 505 is the maximum gradient, these two gradients are both the maximum gradient information of the deformation capability information 113. Must be less than or equal to the specified slope. In order to notify this to the deformable range generation unit 107 that also functions as a control unit, the image deformation unit 104 outputs deformation capability information 113.

  Next, the representative point generation unit 105 receives the input image 100 and outputs representative point information 116. Here, the representative point information 116 is information for the user to control the deformed shape. As shown in FIG. 2, the representative point information 116 includes information on a plurality of representative points, and each representative point information includes current coordinates. Is done. The user controls the deformed shape by selecting and moving the representative point indicated by the representative point information 116. The representative point generation unit 105 receives the input image 100 and generates representative point information 116 from the size of the input image 100. In the four-point designation transformation method of keystone transformation according to the first embodiment, the representative point generation unit 105 outputs four vertices of the input image 100 as the representative point information 116.

  Next, the representative point operation unit 106 performs deformation control by the user, receives the representative point information 116 and the deformable range information 117, and outputs representative point selection information 114 and deformation control information 115.

  Next, the deformable range generation unit 107 receives the representative point information 116, the deformability information 113, and the representative point selection information 114 as input, and generates deformable range information 117. Here, the representative point selection information 114 is information indicating a representative point selected by the user from the representative point information 116. Furthermore, as shown in FIG. 2, the representative point selection information 114 is composed of information on a plurality of selected representative points, and the information on each selected representative point is based on the current coordinates corresponding to the representative point. Composed. Further, the deformable range information 117 includes information on a plurality of selected representative points as shown in FIG. Further, the information of each selected representative point includes coordinates corresponding to each selected representative point and information on the result of the deformability determination.

  The deformability determination result is binary information indicating whether the selected representative point can be deformed at the coordinates. The deformable range information 117 is information (movable range) indicating to which range on the projection plane the representative point selected by the user can be moved. When the user selects one representative point and gives a deformation instruction in the four-point specified deformation method of keystone deformation, the deformable range information 117 is composed of one selected representative point information, and the current of the representative point. Information on the result of determining whether or not deformation is possible in the coordinates around the coordinates.

  Here, the processing of the deformable range generation unit 107 will be specifically described with reference to the processing flow of FIG. When the process starts, the deformable range generator 107 enters a loop that repeats for all selected representative points (step S201). As an example, when the user selects the lower right representative point in the four-point specified deformation method of keystone deformation, there is one selected representative point, and step S201 becomes a single loop. .

  Next, the deformable range generation unit 107 sets the representative point selected as the processing target in step S201 as a movable point (a representative point that can be operated) (step S202). Next, the deformable range generation unit 107 performs a loop process repeated in the X and Y directions on the movable point in the screen coordinate system (XY coordinate system) (step S203). Here, the loop process that repeats in the X and Y directions means a process in which the range of ± dX, ± dY around the selected representative point is used as the coordinate range of the movable point. Here, dX and dY take a large value of the X / Y coordinate distance between the selected representative point and an adjacent representative point or the closest representative display point in the X and Y directions with the selected representative point. In this loop process, the deformable range generating unit 107 sets the coordinates of the movable point to the coordinates specified by the loop that repeats in the X and Y directions (step S204).

  Next, the deformable range generation unit 107 sets coordinates other than the movable point to the coordinates of the current representative point (step S205). In the processing so far, one of the selected representative points is determined as an arbitrary coordinate as a movable point, the other representative points are similarly determined as the coordinates of the current representative point, and the coordinates of all the representative points are determined. It is determined.

  Next, the deformable range generation unit 107 performs a process of determining whether or not the shape specified by the determined representative point can be deformed (step S206). In the first embodiment, since the deformability information 113 is the maximum inclination information, the determination is made based on whether the inclination between the movable point and the adjacent representative point exceeds the maximum inclination indicated by the deformability information 113. As described above, the determination process as to whether or not the deformation is possible is performed based on the positional relationship between the selected representative point that is a movable point and other fixed representative points.

  Next, the deformable range generation unit 107 records the coordinates of the movable point and the deformability determination result (step S207). After completing these loop processes, the coordinates of the movable points corresponding to all selected representative points and the deformability determination result are output as deformable range information 117 (step S208). Above, the process by the deformable range production | generation part 107 is complete | finished.

  In the example of this explanation, a case is described in which the user selects one representative point in the four-point designating transformation method of keystone transformation. Here, when four representative points are selected as the initial state, a loop (step S201) that repeats for all selected representative points is repeated four times. As a result, the deformable range information 117 is generated for the four representative points, and it is possible to display the deformable range for all four representative points before the user selects the representative points.

  Returning to the description of FIG.

  Next, the deformable range drawing unit 108 receives the representative point information 116, the representative point selection information 114, and the deformable range information 117, and outputs a deformable range image 118 in which the input information is drawn. The input / output relationship will be described in more detail with reference to FIGS. The representative point information 116, representative point selection information 114, and deformable range information 117 of each piece of information to be input are as shown in FIG. On the other hand, an example of the output deformable range image 118 is shown in FIG. Here, the process of the deformable range drawing unit 108 will be described in detail with reference to FIGS.

  The deformable range drawing unit 108 draws representative points that are not included in the representative point selection information 114 in the representative point information 116. If the user has selected the lower right representative point (representative point 407 in FIG. 5) in the four-point specified deformation method of keystone deformation, as shown in the deformable range image 118 in FIG. And 406 are drawn. Next, the deformable range drawing unit 108 draws representative points included in the representative point selection information 114. In the case of FIG. 5, the selected representative point 407 is drawn in the deformable range image 118. Next, the deformable range drawing unit 108 draws an area (deformable range) indicated by the deformable range information 117. In this example, in the deformable range image 118 of FIG. 5, the region of the deformable range 408 with respect to the selected representative point 407 is drawn, and the processing of the deformable range drawing unit 108 is completed.

  When the deformability determination result included in the deformable range information 117 indicates that the selected representative point 407 cannot be deformed, this is indicated instead of the deformable range image 118. A predetermined image (for example, a message image) is displayed.

  Next, the image composition unit 109 receives the deformable range image 118 and the deformed image 112 as inputs, and outputs an output image 102 obtained by combining them. This process will be specifically described with reference to FIG. Two examples of the deformable range image 118 and the deformed image 112 are as shown in FIG. 5, and the output image 102 is output by semi-transparently combining the deformable range image 118 on the deformed image 112. However, the image composition method is not limited to the translucent composition, and it is sufficient that the deformable range image 118 can be identified in the output image 102, and a composition process that superimposes the deformable range image 118 may be used.

  As described above, according to the first embodiment, the image deformation device 101 combines the deformable range image 118 including the deformable range 408 of the representative point 407 selected by the user with the deformed image 112 as the output image 102. Output. Thereby, it is notified in advance whether or not the deformed shape intended by the user can be realized in the output image 102. This eliminates the problem of the prior art, in which it is impossible to determine whether the selected vertex can be transformed into the intended shape without actually moving it to the intended shape. It is possible to know in advance whether or not the intended deformed shape can be realized.

  In this way, when the user attempts to deform the projected image by the four-point designated deformation method and place the deformed shape at a predetermined position such as a screen frame, the user can deform the projected image and place it on the screen frame. It becomes possible to know whether or not the transformation is performed. More specifically, when one of the four vertices for performing the deformation operation of the four-point specified deformation method is selected, the deformable range of the selected vertex can be known.

<Embodiment 2>
The second embodiment aims to notify the user of the image quality when the representative point is moved within the deformable range, in addition to the information of the deformable range of the representative point selected by the user. In the image transformation process, image quality deterioration due to generation of jaggies due to image enlargement or image collapse due to reduction occurs. Since these image degradations occur remarkably in a region where the variable magnification (enlargement ratio or reduction ratio) is away from the same magnification, it is desirable for the image quality after deformation to be closer to the same magnification. For this reason, in the second embodiment, the image is generated and output based on the information indicating the image quality in the deformable range of the selected representative point and the scaling factor (enlargement ratio and reduction ratio) of the image after deformation.

  In order to realize the above, the second embodiment is different in the processes of the deformable range generation unit 107 and the deformable range drawing unit 108 in the first embodiment. First, the deformable range generation unit 107 further performs processing for generating image quality information in the determination processing (step S206) of whether or not deformation is possible in the flowchart of FIG. The image quality information is calculated by the following calculation from the maximum magnification and the minimum magnification of the deformed image.

MAX (Maximum magnification-1.0, 1.0-Minimum magnification)
Here, the MAX () function is a function that returns the larger value of the arguments. The purpose of this equation is to obtain the larger one of the absolute values of the difference between the same magnification (1.0) and the maximum magnification and the minimum magnification. This is based on the idea that although the enlargement or reduction occurs due to the deformation, the image quality is likely to be lowered more distant from the same magnification. If the maximum magnification is 1.1 (110%) and the minimum magnification is 0.8 (80%), then 0.2 (= MAX (1.1-1.0, 1.0-0. 8)) is obtained.

  The above-described value is associated with image quality information defined in advance, and image quality information is output. As a specific example, 0.0 to 0.1 is image quality: good, 0.1 to 0.2 is image quality: medium, and 0.2 to is image quality: low. However, the association is not necessarily as described above, and the image quality that is close to the same magnification is good and the image quality that is far from the normal magnification is associated with low image quality. The image quality information generated in this way is output in addition to the deformable range information 117.

  Next, the deformable range drawing unit 108 draws the deformable range image 118 using the deformable range information 117 including the image quality information. A specific example is shown in FIG. In the deformable range image 118 of FIG. 6, in addition to the deformable range of the selected representative point 604, the image quality information includes a range 605 where the image quality is “good”, a range 606 where the image quality is “medium”, and an image quality “low”. Is output as a range 607. The deformable range image 118 generated in this way is combined with the deformed image 112 and output by the image combining unit 109, and the output image 102 is generated.

  As described above, according to the second embodiment, the image deformation device 101 can notify the user of the image quality in addition to the information on the deformable range of the representative point selected by the user. Thus, the user can know in advance the degree of degradation in image quality when the selected vertex is moved to an arbitrary position before actually performing the deformation operation.

  That is, when the user selects any of the four vertices for performing the deformation operation of the four-point specifying deformation method, the user can know the image quality when the selected point is operated and the deformation is specified.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (14)

Deformation means for executing deformation processing for generating a deformation image from an input image;
Designating means for designating the coordinates of the input image;
Generating generated based on the positional relationship between the other coordinates of the range image showing the movable range due to the deformation processing by the deformation means of the designated coordinates, and the designated coordinate the input image by the specifying means Means,
Projection control means for projecting the range image generated by the generating means onto the projecting means;
An image processing apparatus comprising: Before Symbol generating means, the positional relationship of the the specified coordinates and other coordinates on the input image, and generates said range image based on the size of the memory for storing the input image The image processing apparatus according to claim 1. Before Symbol generating means, the positional relationship between the other coordinates of the input image and the specified coordinates, and, according to claim 1, characterized in that to generate the range image based on the resolution of the input image Image processing apparatus. The image processing apparatus according to claim 1 , wherein the designating unit designates coordinates selected by a user from a plurality of predetermined coordinates among coordinates on the input image. Said projection control means, said range image out of claims 1 to 4 and the deformed image is characterized in that projecting the synthesized image synthesized in the projection unit, an image processing apparatus according to any one . The designation means designates one of four vertices corresponding to the input image,
The said production | generation means produces | generates the said range image based on the positional relationship of the said designated vertex and at least one of the remaining vertices, The any one of Claims 1 thru | or 5 characterized by the above-mentioned. Image processing apparatus. When it is determined that the deformation process based on the movement of the designated coordinates on the input image is impossible, the projection control unit replaces a predetermined image indicating the impossibility of the deformation process with the range image. The image processing apparatus according to claim 1 , wherein the image processing apparatus projects the image. A specifying means for specifying the image quality after the deformation process based on a scaling factor due to the deformation of the input image;
Said generating means, among the claims 1 to 7, characterized in that to generate the range image including information indicating the image quality specified by the specifying unit, an image processing apparatus according to any one. The deforming means, of the claims 1 to 8, characterized in that to perform the transformation process in accordance with the deformation amount corresponding to the user input, the image processing apparatus according to any one. An image processing method by an image processing apparatus having deformation means for executing deformation processing for generating a deformation image from an input image,
An accepting step of accepting designation of coordinates of the input image;
A range image for indicating a movable range based on the deformation process by the deformation unit of the coordinates received in the reception step is generated based on a positional relationship between the designated coordinates and other coordinates of the input image. Generation process;
A projection control step of causing the projection means to project the range image generated by the generation step;
An image processing method comprising: Before Symbol generation step, the positional relationship between the other coordinates of the input image and the specified coordinates and the generating means generates the range image based on the size of the memory for storing the input image The image processing method according to claim 10 . Before Symbol generation step, the positional relationship between the other coordinates of the input image and the specified coordinates, and, according to claim 10, characterized in that to generate the range image based on the resolution of the input image Image processing method. In the receiving step, one designation of four vertices corresponding to the input image is received,
The range image is generated in the generation step based on a positional relationship between the designated vertex and at least one of the remaining vertices. An image processing method described in 1. A program for causing a computer to operate as each unit of the image processing apparatus according to any one of claims 1 to 9 .

Images