JP2005293616A - Image synthesizer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily create a composite image without a sense of incongruity without special experience or intuition when a texture image is synthesized with an original image.
In an image composition device for compositing a texture image to a part of an original image, an image input device 10 for inputting the original image and the texture image, and a composition region for compositing the texture image in the original image are designated. An instruction input device 12 for designating a parameter for projectively normalizing the coordinate value of the composite area to a coordinate value on the texture image, and generating mask data representing the composite area Projective normalization conversion is performed on the coordinates in the composite region based on the determined parameters to calculate a relative coordinate value, a texture coordinate value on the texture image is calculated from the relative coordinate value, and the texture coordinate value And an image synthesis device 14 for synthesizing a texture image in the synthesis area.
[Selection] Figure 1

Description

  The present invention relates to an image synthesizing apparatus that is suitable for application when a synthesized image is created by synthesizing another texture image with a desired region in an original image.

  In the catalog of products such as building exterior materials and interior materials, kitchen / storage doors, etc., actual construction examples may be listed only for some products, and only other color / material samples may be listed for other products. Many customers had to look at actual construction examples and samples, imagine the appearance of applying the desired samples to the construction examples, and select products.

  Thus, in the catalog, since there is no actual construction example for each product, there is a problem that it is difficult for a customer to grasp an image in order to select a product. The above problem can be solved by actually creating construction examples and catalogs for all products, but this is too expensive and unrealistic.

  As another solution, there is a method of synthesizing other product sample images with images of actual construction examples, but in order to obtain a composite image without any sense of incongruity, a skilled operator uses trial and error using experience and intuition. However, it is difficult to perform this operation for many sample products, and the cost is increased.

  The present invention has been made to solve the above-mentioned conventional problems, and has special experience and intuition when a texture image such as a product sample image is synthesized with an original image such as a construction example of an arbitrary product. It is an object of the present invention to provide an image composition device that can easily create a composite image without any sense of incongruity.

  The present invention relates to an image synthesizing apparatus for synthesizing a texture image with a part of an original image, means for inputting an original image, means for inputting a texture image to be synthesized with the original image, and a texture image among the original images Means for designating a synthesis region for synthesizing the image, means for designating a parameter for coordinate conversion of the coordinate value of the synthesis region to a coordinate value on the texture image, and the coordinates in the synthesis region based on the parameter Means for calculating a relative coordinate value by performing coordinate transformation, means for calculating a texture coordinate value on the texture image from the relative coordinate value, and synthesis for synthesizing a texture image in the synthesis area based on the texture coordinate value The above-mentioned problem is solved by adopting a configuration including means.

  In the present invention, a parameter for coordinate conversion of the coordinate value of the synthesis area designated on the original image to the coordinate value on the texture image is designated, and coordinate transformation is performed on the coordinates in the synthesis area based on the parameter. The relative coordinate value is calculated, the texture coordinate value on the texture image is calculated from the relative coordinate value, and the texture image is synthesized in the synthesis area based on the texture coordinate value. By using the correspondence relationship of the coordinate system, it is possible to synthesize a texture image of an arbitrary size in the synthesis area and create a synthesized image without a sense of incongruity.

  According to the present invention, when a texture image such as a product sample image is synthesized with an original image such as a construction example of an arbitrary product, a composite image without a sense of incongruity can be easily created without special experience or intuition. can do.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an image composition system (image composition apparatus) according to an embodiment of the present invention. The image composition system of this embodiment includes an image input device 10, an instruction input device 12, an image composition device 14, a display device 16, and an image output device 18.

  The image input device 10 includes an image scanner that inputs an original image, a texture image to be synthesized with the original image, and a storage device such as a hard disk in which these image data can be stored and read. The instruction input device 12 is composed of a keyboard, a mouse, and the like for designating a synthesis area for synthesizing a texture image in the original image, and for converting the coordinate value of the synthesis area into a coordinate value on the texture image. You can specify the parameters.

  The image synthesizing device 14 is composed of a computer, and the device 14 includes means for performing coordinate transformation on coordinates in the synthesis region based on designated parameters and calculating relative coordinate values, and the relative coordinate values. The means for calculating the texture coordinate value on the texture image and the means for synthesizing the texture image in the synthesis area based on the texture coordinate value are realized by software. The display device 16 includes a display such as a CRT or a liquid crystal display, and displays an original image, a texture image, a composite image, and the like. The output device 18 includes a printer or the like, and can output a composite image. It is like that.

  More specifically, the image synthesizing device 14 uses the same software to generate original image luminance data consisting only of luminance information of each pixel of the original image, and the luminance of the synthesized texture image. A luminance changing means for changing the original image luminance data and a parameter for controlling the difference between light and dark is realized. Similarly, mask data created in consideration of the scene-blocking area is applied to the synthesis area, and the parameters used for the coordinate transformation are the original that is a rectangle in an actual three-dimensional space. Four vertex coordinates for defining a rectangular area on the image are included, and the means for calculating the relative coordinate value can perform coordinate conversion based on the parameters in consideration of the mask data. ing.

  Similarly, the parameters used for the coordinate conversion include the actual size (Sx, Sy) of the quadrilateral representing the composite area on the original image that is rectangular in the actual three-dimensional space. ), And the synthesis unit repeats the texture image a number of times according to the relative ratio between the actual size (Sx, Sy) of the rectangle and the actual size (Strx, Stry) of the texture image. And can be synthesized in the synthesis region. Further, a plurality of the texture images are prepared, and the means for calculating the texture coordinate value is obtained by selecting the texture coordinate value based on the relative coordinate value already obtained when another new texture image is selected. It can be calculated.

  Next, the operation of this embodiment will be described with reference to the flowchart shown in FIG.

  First, an original image (FIG. 3) and a texture image (FIG. 4) are input by an image input device (steps 1 and 2). In FIG. 3, the two-dot chain line attached to the front surface of the rear object (cube) indicates the boundary because it is difficult to identify the shadow generated by the front cube.

  Next, mask data representing a region where the texture image is synthesized is created (step 3). This mask data represents the effective area in which the texture is synthesized in the original image. Each area where different textures are synthesized, or how the perspective is applied, that is, the actual tertiary of the original image. A different identification number is set for each region that is not on the same plane when the original space is considered.

  As an example of this mask data, array data having the same coordinate values as the original image in the vertical and horizontal dimensions is prepared, and a value of 0 is set for pixels that are not subjected to texture synthesis, and pixels for which texture synthesis is performed. Is set with the aforementioned identification number. Here, as an example of the mask data creation procedure, an image in which only the composite area is processed into an arbitrary color (here, white) as shown in FIGS. 5 and 6 based on the original image is prepared. Mask data as shown in FIG. 7 was created from the image. Here, for the sake of convenience, an area where synthesis is not performed is shown in white, and a different color (black and gray) is shown for each effective area. Further, in the present embodiment, in the original image of FIG. 3, in the case of a rear cube in which a scene is blocked by the cube in front, the effective area is shown in black in FIG. Then, mask data in a range excluding the light shielding area is created.

  Next, a parameter for projectively normalizing the coordinate value of the synthesis area on the original image to synthesize the texture image to the coordinate value on the texture image is designated (input) (step 4). In this operation, the coordinate value of the composite area on the original image of each of the vertices P1 to P4 of the rectangle on the same plane in the actual three-dimensional space and the x direction of the rectangle in the actual three-dimensional real space , Y direction sizes Sx and Sy are designated by the instruction input device 12. In this case, if the actual size of the rectangle is known or can be estimated, the value may be specified. However, the size of the rectangle is not known, and is estimated from a known size in the original image. If this is not possible, there is no problem even if relative values such as how many texture images are arranged in the vertical direction and how many in the horizontal direction are used in the region surrounded by the designated vertices P1 to P4. It is only necessary to specify 1 when designating the actual sizes Strx and Stry of the texture image to be described later.

  Here, supplementing the coordinates (parameters) of the four vertices to be designated, these coordinate values are obtained by combining the composition area itself to be synthesized or the mask area (effective area defined by the mask data) as shown in FIG. Needless to say, the mask area is not directly related to the mask area, and if it is an area on the original image that is the same plane in the actual three-dimensional space, that is, if the method of parsing is the same, the mask area There is no problem if it is specified outside, and there is no problem if it is specified outside the area of the original image.

  As a specific example of parameter specification, FIG. 8 shows an example in which coordinate values are specified for a mask area corresponding to the composite area shown in FIG. In this figure, coordinates (x1, y1) to (x4, y4) of P1 to P4 are designated at the vertices of a rectangle on the original image which is a rectangle in an actual three-dimensional space, and the actual three-dimensional space of this rectangle is displayed. The sizes Sx and Sy in the x and y directions are specified. As for P2, it is hidden by an object in front, and this vicinity (blocking area) is outside the effective area in the mask data shown in FIG. 7, but parameter designation is not directly related to the mask area as described above. Any point on an area that will be rectangular in an actual three-dimensional space can be used without any problem. It is necessary to specify the above parameters for all mask areas to be combined. However, if the parameters are the same plane in the actual three-dimensional space, that is, the areas on which the perspective is applied are the same on the original image. The same parameters can be applied.

  Next, various arithmetic processes described in detail below are performed by the image synthesizer 14. First, a projective normalization transformation is performed on the coordinates in the synthesis area based on the above parameters (this coordinate transformation is described in detail in, for example, “Computer Geometry and Geographic Information Processing” bit separate volume Kyoritsu Publishing (1986)). Do. Specifically, with respect to all pixels included in the area to be textured to be described later, the expressions (3) and (4) including c and d defined by the following expressions (1) and (2) are used. Then, projective normalization conversion is performed to calculate relative coordinate values α and β (step 5). Here, the relative coordinate value is given as a value normalized to rectangular coordinates composed of actual sizes Sx and Sy in the x and y directions.

  In the calculation of the relative coordinate values in step 5, the coordinates are converted by the above equations (1) to (4) using the parameters of the vertex coordinates (x1, y1) to (x4, y4) and the actual sizes Sx and Sy. In this case, the processing is performed in consideration of the mask data (mask region). As a result, if there is a shaded area, there is no need to perform coordinate conversion calculation processing for that range, so the amount of calculation can be reduced and the processing speeded up.

Next, a texture image to be synthesized is selected (designated) from those already input by the instruction input device 12 (step 6), and a synthesis area (one area or a plurality of areas may be synthesized) to synthesize the texture image. Is designated (step 7). At this time, when the texture image to be synthesized is reduced and synthesized, it is necessary to perform low-pass filter processing or the like on the texture image in advance to suppress the occurrence of moire.

  Next, the actual size of the texture image to be synthesized is designated (as described above, 1 is designated when the size of the rectangle is designated by a relative value indicating how many texture images are arranged). Then, the relative coordinate values (α, β) calculated by performing the projective normalization conversion on the coordinate values in the synthesis area to be synthesized are expressed in terms of the actual size of the texture image and the pixel unit of the texture image. Based on the size, texture coordinate conversion is performed by converting the coordinate value (xt, yt) on the texture image to be actually synthesized according to the following equation (step 8). At that time, when xt and yt are less than 0 (when α and β are negative in the relative coordinate system as shown in FIG. 11 described later) or greater than Stpx and Stpy, the remainder obtained by dividing by Stpx and Stpy is calculated. The coordinate value on the texture image is set with the number of remaining pixels.

  Next, the pixel value of each pixel constituting the texture image of the coordinate converted in step 8 is selected by replacing the pixel value of the pixel to be synthesized (the corresponding pixel in the synthesis area in the mask area). A texture image is synthesized in the synthesis area (step 9). At that time, the actual size (Sx, Sy) of the quadrilateral representing the composite area on the original image which is a rectangle in the actual three-dimensional space and the actual size of the texture image (Strx). , Stry) and the texture image is repeated and synthesized in the synthesis area for the number of times given as the relative ratio obtained by Details of this repeated synthesis will be described later for convenience.

  Next, the brightness of the synthesized texture image is changed based on the brightness data of the original image (step 10). Specifically, the luminance is changed by the following equation (7) for each pixel in the region where the texture image is synthesized by the synthesis process in step 9 above. This makes it possible to increase or decrease the difference in light and darkness in the synthesis area of the original image, or to increase or decrease the average brightness difference between the original image and the synthesized texture image to obtain a more natural synthesized image. Can be done. Here, by changing each parameter, it is possible to obtain a natural composite image without a sense of incongruity even when it is desired to synthesize extremely different texture images such as the material on the composite region of the original image and the reflectance. If there are other texture images to be synthesized, the above-described processing in steps 6 to 10 is repeated (step 11).

  FIG. 9 shows a composite image created by actually setting the mask data of FIG. 7 on the original image of FIG. 3 and synthesizing the texture image of FIG. 4 in the two areas. In FIG. 9, it is difficult to understand because the gradation difference is not clear, but the shadow indicating the boundary with a two-dot chain line is reflected in the composite image as in FIG. 3.

  Next, for convenience, in calculating the coordinate values on the texture image as described above, the relative coordinate values α and β are calculated once, and then the reason why xt and yt are calculated is described with reference to the drawings. This will be described in detail.

  Now, if the actual size of the composite area shown in FIG. 10 is, for example, m in width 5 and height 3, the coordinates after conversion to α and β are as shown in FIG. 11 (here Can be calculated regardless of the actual size of the texture in pixel units). Here, in this area, the texture image schematically shown in FIG. 12 (size in pixel units: 100 × 100, actual size: 1 × 1) is converted into coordinate values on the texture image. The results obtained are shown in FIG. 14 as the result of repeatedly synthesizing the texture image. In addition, the result of converting the texture image schematically shown in FIG. 13 (size in pixel units: 150 × 150, actual size: 2 × 2) into coordinate values on the texture image is shown. FIG. 15 shows the result of repeated synthesis.

  As is clear from this example, the actual size of the P1 to P4 region, the size of the texture image in pixel units, and the actual size can be specified, so that the actual size can be synthesized. Is possible. In addition, once α and β are calculated, α and β can be calculated regardless of the size of the texture image. Therefore, even when a texture image having a different actual size is synthesized in units of pixels, α, β Since the value of can be reused, the amount of calculation can be reduced.

  Although the present invention has been specifically described above, the present invention is not limited to that shown in the above embodiment, and various modifications can be made without departing from the scope of the invention.

For example, the projective normalization conversion is adopted as the coordinate conversion in the embodiment, but the present invention is not limited to this, and the coordinate conversion that normalizes the synthesis area on the original image into a rectangle (including a square) in the actual three-dimensional space. If it is, it will not be restricted. Further, the specific configuration of the image composition apparatus is not limited to that shown in FIG.

1 is a block diagram showing a schematic configuration of an image composition apparatus according to an embodiment of the present invention. A flowchart showing the operation of the embodiment Explanatory drawing showing an example of the original image Explanatory drawing showing an example of a texture image Explanatory drawing showing the creation procedure of mask data Another explanatory diagram showing the procedure for creating mask data Explanatory drawing showing an example of created mask data Explanatory diagram showing how to specify parameters Explanatory drawing which shows an example of a composite image Diagram showing an example of a composite area with parameters set Diagram showing the above-mentioned composite area converted to relative coordinates Diagram showing an example of texture image Diagram showing another example of texture image Diagram showing an image synthesized from the texture image of FIG. Diagram showing an image synthesized from the texture image of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image input device 12 ... Instruction input device 14 ... Image composition device 16 ... Display device 18 ... Image output device

Claims (2)

In an image synthesizer that synthesizes a texture image with a part of an original image,
Means for inputting the original image;
Means for inputting a texture image to be combined with the original image;
Means for designating a synthesis region for synthesizing a texture image in the original image;
Means for designating a parameter for coordinate conversion of the coordinate value of the synthesis area into a coordinate value on the texture image;
Means for performing coordinate transformation on the coordinates in the composite region based on the parameters and calculating relative coordinate values;
Means for calculating a texture coordinate value on the texture image from the relative coordinate value;
An image synthesizing apparatus comprising: a synthesizing unit that synthesizes a texture image in the synthesis region based on the texture coordinate value. In claim 1,
Means for generating original image luminance data consisting only of luminance information of each pixel of the original image, and luminance change for changing the luminance of the synthesized texture image based on the original image luminance data and a parameter for controlling the difference between light and dark And an image synthesizing apparatus.

Images