JPH06274650A - Image synthesizer - Google Patents

Abstract

PURPOSE:To prepare a synthetic image with the natural feeling of depth between a texture image synthesized with a source image and another source image area without depending on the shape of a synthetic area. CONSTITUTION:An image input device 10 inputs the source image and the texture image, an image area designating device 12 designates the perspective designating area of the source image and the using area of the texture image by respectively inputting apexes on coordinates, and an image synthesizer 14 sets a mask to specify the synthetic area in the perspective designating area based on data read from the image input device 10 and the image area designating device 12, performs perspective to the using area of the texture image by performing inverse projective normalizing transformation opposite to coordinate transformation for calculating projective normalizing coordinates, synthesizes the images by sticking the perspective texture image into the perspective designating area and outputs the prepared synthetic image from the image synthesizer 14 to an image output device 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
For example, in the production of presentation images, electronic platemaking, etc., when a texture image is pasted onto an original image and image synthesis is performed, it is possible to give the texture image after synthesis a natural sense of depth that is consistent with the original image. Regarding the device.

[0002]

2. Description of the Related Art For example, when a texture image is pasted on an original image to synthesize an image in the production of a presentation image or electronic plate making, the original image is created by a perspective drawing method, which gives a perspective. If it is given, if the texture image is pasted without distortion to give a sense of perspective (hereinafter also referred to as perspective) to the texture image to be pasted into the composite area of the original image, Since there is no sense of depth in the texture image that has been embedded, in the resulting composite image, a sense of incongruity (difference in depth of perspective) may occur between the region where the texture image is pasted and other regions due to inconsistency in depth. become.

As a method of distorting the texture image in order to eliminate such a feeling of discomfort, there is a method of setting a composite area of a predetermined shape in the original image and converting the coordinates of the texture image according to the shape of the composite area. is there.

[0004]

However, the above-mentioned method of coordinate-converting the texture image in accordance with the shape of the composite area has the following problems. Now, with respect to an original image in which a triangular window as shown in FIG. 20 when viewed from the front is provided in a room with a perspective as shown in FIG. Consider a case where the texture image T as shown in FIG. 21 is subjected to coordinate conversion and is combined with the window portion.

At that time, when the area (composite area) corresponding to the triangular window in FIG. 19 is used as it is as a reference area (perspective designated area) for giving a sense of perspective to the texture image T, the texture image T in FIG. 4 vertices P1 ', P2', P3 ', P4' that define the use area of
Are the four vertices P1, P2, P3,
It corresponds to the perspective designated area defined by P4, but since this is a triangle, the vertices P1 and P2 have the same coordinate values. As a result, the texture image transformed by the coordinate transformation (perspective transformation) is reduced in the height direction and the depth direction (horizontal direction) as it goes to the left side of the window as seen in FIG. Will be obtained.

The present invention has been made in order to solve the above-mentioned conventional problems. Whatever shape the composite area of the original image has, the original image is pasted in the composite area without being affected by the shape. An object of the present invention is to provide an image synthesizing device capable of producing a synthetic image having a natural sense of depth without a sense of discomfort between a texture image and other regions of an original image.

[0007]

According to the present invention, a means for inputting an original image, a means for inputting a texture image, a means for designating a pasting area for pasting the texture image on the original image, and a pasting for the original image. In an image synthesizing device including means for designating a used area of a texture image corresponding to an embedded area and image synthesizing means for synthesizing a used area of a texture image with a pasted area of an original image, By having a function of setting a mask having at least one window area in the pasting area of, and a function of performing pseudo perspective transformation on the usage area of the texture image,
The above object has been achieved.

According to the present invention, in the image synthesizing apparatus, the pseudo perspective transformation function provided in the image synthesizing means is a function for performing projective normalization inverse transformation.

[0009]

In the present invention, the pasting area (paste designated area) for pasting the texture image on the original image is designated, and the used area of the texture image is pseudo-perspectively converted in correspondence with the perspective designated area. By creating a parsed texture image by doing this and pasting the parsed and deformed texture image into the parse designated area, a mask having at least one window area can be set in the parse designated area. Therefore, it is possible to independently set the perspective designated area in which the deformed texture image is to be pasted and the composite area (mask window area) in which the texture image is actually composited.

As a result, the pseudo perspective transformation can be applied to the texture image without being affected by the shape of the composite area, so that, for example, between the outside scene seen through a window of an arbitrary shape and the original image. This makes it possible to create a synthetic image that does not look strange.

Further, in this case, when a mask having a plurality of window regions is used, for example, the outside scenery seen from the room through the plurality of windows maintains a continuity, and the windows have a sense of depth without any discomfort. It is possible to create images that have been combined with.

In the present invention, the perspective designated area is a quadrangle set in the original image in order to control the degree of distortion applied to the texture image, and each vertex has four vertices of the texture image used area. Are areas to which projective normalization inverse transformation is applied so as to correspond to.

The pseudo perspective transformation is a purely two-dimensional method (without using the three-dimensional depth value information) and uses a geometrical transformation means to give a pseudo depth feeling to an image. It means a process to have.

Further, in the present invention, when the pseudo perspective transformation function provided in the image synthesizing means is a function for performing the projective normalization inverse transformation, the space between the synthesized texture image and another area is In addition, it is possible to produce a composite image with an extremely natural sense of depth without any discomfort.

The projective normalized inverse transformation will be described in detail below. First, the projective normalized coordinate transformation which is the basis of the projective normalized inverse transformation will be described. Regarding this coordinate transformation, for example, "Computational Geometry and Geographic Information Processing" bit separate volume Kyoritsu Publishing (1986) edited by Masao Iri / edited by Takeshi Koshizuka, and "Basic Knowledge of Numerical Information" (1981) Geographical Survey of the Ministry of Construction It is explained in detail in the institution map information room.

The projective normalized coordinate transformation is a coordinate transformation suitable for correcting aerial photographs and has the following features. now,
If there is a point P inside the quadrangles P1, P2, P3, and P4, P1 (x ₁ , y ₁ ) becomes P1 ′ (0, 0), P2
Let (x ₂ , y ₂ ) be P2 '(0, 1), P3 (x ₃ ,
y ₃ ) to P3 '(1,1) and P4 (x ₄ , y ₄ ) to P
P (x, by projective transformation that moves to 4 '(1,0)
When the point to which y) shifts is P '(α, β), this coordinate (α, β) is called projective normalized coordinate, and the projective transformation for obtaining this coordinate is defined as projective normalized coordinate transformation. To do.

The projective transformation for associating Pi and Pi '(i = 1, 2, 4) is expressed by the following equations (1) and (2).

[0018] _{x - x 1 = {(1} + c) (x 4 - x 1) α + (1 + d) (x 2 - x 1) β} ÷ (1+ cα + dβ) ......... (1) y - y 1 = { _{(1 + c) (y 4} - y 1) α + (1 + d) (y 2 - y 1) β} ÷ (1+ cα + dβ) ......... (2)

Here, when α = β = 1, x = x ₃ ,
If c and d are determined so that y = y ₃ , the following equation (3),
Equation (4) is obtained.

[0020]

[Equation 1]

Δ ₁ = (x ₄ −x ₃ ) (y ₂ −y ₃ ) − (y ₄ −y ₃ ) (x ₂ −x ₃ ) ... (4)

By solving the equations (1) and (2) for (α, β), the following equations (5) and (6) are obtained.

[0023]

[Equation 2]

Δ ₂ = {x ₄ −x ₁ −c (x −x ₄ )} {y ₂ −y ₁ −d (y −y ₂ )} − {y ₄ −y ₁ −c (y −y _{4 )} {x 2 - x 1} -d (x - x 2)} ......... (6)

That is, for the projective normalized coordinates (α, β) of P (x, y), c and d are obtained by the equations (3) and (4), and then the equations (5) and (6) are obtained. It can be calculated by calculating (α, β).

As is clear from the equation (3), c, d
Is determined by quadrangles P1, P2, P3, and P4, and when it is a parallelogram, c = d = 0.

When one quadrilateral P1, P2, P3, P4 is fixed and many projective normalized coordinates of P (x, y) are obtained, c and d can be calculated in advance.

The projective normalization inverse transformation applied to the present invention is a coordinate transformation for obtaining a distorted image from a normalized image by reversely utilizing the above-mentioned projective normalization coordinate transformation. is there.

That is, this projective normalization inverse transformation is performed by using the normalized parameters α, β (0 ≦ α ≦ 1), (0 ≦ β ≦
The position to which the coordinate value (α, β) represented by 1) corresponds to a quadrilateral having an arbitrary shape is obtained as follows.

Now, considering a quadrangle as shown in FIG. 1, the inverse transformed coordinate values (x, y) obtained by projective normalized inverse transformation of the normalized coordinate values (α, β) are the same as the above ( Formula 1), (2)
From the equations, the following equations (7) and (8) are obtained.

X = x ₁ + {(1 + c) (x ₄ −x ₁ ) α + (1 + d) (x ₂ −x ₁ ) β} ÷ ( ₁ + cα + dβ) (7) y = y ₁ + { _{(1 + c) (y 4} - y 1) α + (1 + d) (y 2 - y 1) β} ÷ (1+ cα + dβ) ......... (8)

In the above equations (7) and (8), c and d
Is the same as in the case of the projective normalized coordinate transformation (3)
Equation is given by equation (4).

Next, the effect of the above-described projective normalization inverse transformation is a perspective transformation in which the transformation for obtaining the normalized coordinates from the measurement coordinates used in geographic information processing is executed in reverse (hereinafter referred to as the normalization inverse transformation). Will be specifically described in comparison with the case of applying.

Now, as shown in FIG. 2 (A), a trapezoidal original image pasting area (perspective designation area) D having two sides parallel to each other is arranged in a grid shown in FIG. 2 (B). Consider a case where the texture image T is parsed in order to paste the texture image T.

When the texture image T of FIG. 2B is parsed by the above-mentioned normalization inverse transformation, the vertical direction (vertical direction) of the pasting area becomes vertical as shown in FIG. Although it is possible to apply the distortion of the texture correspondingly, since the lateral direction (horizontal direction) remains at equal intervals, the front side has a shrunken impression and a natural depth feeling cannot be obtained.

On the other hand, the above projective normalization inverse transformation is
When applied to the lattice-shaped texture image T shown in FIG. 2B that is pasted in the perspective designated area D shown in FIG. 2A, a parsed texture image T as shown in FIG. 4 can be obtained. it can.

In the texture image T shown in FIG. 4,
It is possible to apply texture distortion in the vertical direction corresponding to the length (vertical direction) of the perspective-designated area, and to apply distortion in the horizontal direction (horizontal direction) according to the vertical distortion.

Therefore, according to the projective normalized inverse transformation, as compared with the texture image after transformation shown in FIG. 3 obtained by applying the above-mentioned normalized coordinate transformation to the texture image in the image editing work, It can be seen that it is possible to create a composite image having a more natural depth feeling.

[0039]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 5 is a block diagram showing the schematic arrangement of the image synthesizing apparatus of the first embodiment according to the present invention.

The image synthesizing apparatus according to the present embodiment includes an image input apparatus 10, an image area designating apparatus 12, and an image synthesizing apparatus 14.
And an image output device 16.

The image input device 10 has a function of inputting an original image and a texture image to be combined with the original image, and the image area designating device 12 has an original image pasting area (a perspective designating area). ) And the use area of the texture image are specified by inputting the vertices on the coordinates.

The image synthesizing device 14 is the image input device 1.
0 and data is read from the image area designating device 12, and a function is provided for setting a mask having a window area for the perspective designated area designated in the original image based on the data, and by the method described in detail later. The texture output has a function of performing projective normalization inverse transformation and synthesizing the texture image after the transformation and the original image to synthesize an image with a natural sense of depth. Has a function of outputting the combined image to, for example, a file.

Next, the operation of this embodiment will be described with reference to the flow charts shown in FIGS.

First, the image input device 10 inputs an original image and a texture image (step 110), specifies a perspective designated area for the input original image, designates a used area of the texture image, and A mask is set for the perspective designated area of the image (step 112).

Here, as shown in FIG. 8, with respect to the texture image T, an area used for image synthesis is a quadrangle P.
1 ', P2', P3 ', P4', and the use area is specified by the coordinates of these four vertices. Further, as shown in FIG. 9, a perspective designated area D on the original image G corresponding to the texture image T is defined by quadrilaterals P1, P2, P3 and P4, and the designated area D is similarly defined by these four areas. Specify with the coordinates of the vertex.

Next, the coordinate value (u, v) is obtained by projective normalization coordinate conversion for one pixel (x ₀ , y ₀ ) of the original image (step 114). This coordinate value (u, v) is
It can be obtained by directly applying the above-mentioned projective normalized coordinate transformation. Here, the projective normalized coordinates for the quadrangle P1, P2, P3, P4 of the perspective designated area D are obtained. In addition, the coordinates (u, v) are expressed by the equation (1),
This corresponds to the normalized coordinate value (α, β) in the equation (2).

Then, the normalized coordinate value (u, v)
Coordinate value (x _t, Y_t) Ask
(Step 116). This coordinate value (x_t, Y_t) Is before
Note that the projective normalized coordinates (α, β) are added to the equations (7) and (8).
Substituting (u, v) as above, the quadrilaterals P1 ', P
Perform calculations to find the coordinates in 2 ', P3', P4 '
Required by. At this time, the projective normalized coordinates are given.
In the obtained quadrilateral P1 ', P2', P3 ', P4'
Calculate the absolute coordinate value.

Next, the coordinates (x ₀ , y ₀ ) on the original image G
The pixel value of is replaced with the pixel value of the coordinate (x _t , y _t ) on the texture image T (step 118).

The above-mentioned steps 114 to 118 are carried out for all the pixels in the perspective designated area D of the original image (step 120).

In the processing from step 114 to step 120, the use area of the texture image T which has been parsed corresponding to the perspective designated area D is obtained by subjecting the use area of the texture image T to projective normalization inverse transformation. Corresponds to the synthesis of the perspective designated area D.

After performing the above processing, the image synthesizing device 1
4 outputs the created composite image to the image output device 16.

According to this embodiment described in detail above, a mask having a window area of a predetermined shape is set for the perspective designated area of the original image, and the entire usage area of the texture image corresponds to the entire perspective designated area. By adding and performing image composition, it is possible to paste a texture image that is appropriately parsed in the parse designated area to the original image, and therefore, regardless of the shape of the composition area (window area). In addition, it is possible to create a composite image having a natural sense of depth in which the texture image after composition is matched with other original image areas without being affected by its shape.

Next, the original image G shown in FIG.
This embodiment will be described more specifically by taking the case of synthesizing the texture image T shown in FIG.

First, step 11 of the above flow chart.
At 0, the original image G of FIG. 10 and the texture image T of FIG. 21 are input by the image input device 10, and at step 112, the original image G as shown in FIG. 12 is specified, and the use area of the texture image T is specified, and the mask M is set for the perspective specified area D of the original image G as shown in FIG. Here, the used area of the texture image has the same size as the input texture image T,
The mask M has a window region corresponding to the triangular window shown in FIG.

Next, the perspective designated area D of the original image G is specified.
And the texture image T, the steps 114-
By executing the process of 120, the texture T is parsed by projective normalization inverse transformation and is also pasted in the parse designated region D.

By the above processing, the perspective designating area D serving as a reference for parsing the texture image T and the mask M for designating the composite area can be set independently of each other. It is possible to perform various parsing and synthesize the desired shape. As a result, as shown in FIG. 13, it is possible to produce a composite image in which the scenery seen from the triangular window has a natural sense of depth. The two-dot chain line in the figure shows the texture image shielded by the mask M.

Next, an image synthesizing apparatus according to the second embodiment of the present invention will be described.

In the image synthesizing device of this embodiment, the image synthesizing device 14 shown in FIG. 2 can set a mask having a plurality of window regions for the perspective designated region D designated in the original image G. The image synthesizing apparatus is substantially the same as the image synthesizing apparatus according to the first exemplary embodiment except that the image synthesizing apparatus has the function.

The operation of this embodiment will be described with reference to the flowcharts of FIGS. 14 and 15. Note that, here, similar to the specific example shown in the first embodiment, the case where the texture image T shown in FIG. 21 is combined with the original image G shown in FIG. 10 will be described as an example.

First, the original image G of FIG. 10 and the texture image T of FIG. 21 are input by the image input device 10 (step 130), and the original image G is designated by the image area designating device 12 as shown in FIG. The perspective designated area D and the usage area of the texture image T are designated, and a mask M having two window areas is set for the perspective designated area D of the original image G as shown in FIG. 16 (step 13).
2). In this embodiment, the use area of the texture image and the purse designation area of the original image can be designated by using the vertex coordinates of each quadrangle as in the first embodiment.

Next, the texture image T shown in FIG. 21 is subjected to a perspective transformation corresponding to the perspective designated area D (a texture transformed image) (corresponding to the texture image T shown in FIG. 17).
And one pixel (x ₀ ,
coordinate value by the projective normalization coordinate transformation for y ₀₎ (u
, V) is obtained (step 134).

The coordinate values (u, v) can be obtained by directly applying the above-described projective normalized coordinate transformation, as in the case of step 114 in the first embodiment. That is, the projective normalized coordinates for the quadrangle of the texture deformed image are obtained.

Then, the normalized coordinate values (u, v) are paired.
Corresponding coordinate values on the texture image T shown in FIG.
(X_t, Y_t) Is obtained (step 136). This coordinate
Value (x _t, Y_t) Is the step 136 of the first embodiment.
As in the case of, the above equations (7) and (8)
Substituting (u, v) above as the normalized coordinates (α, β)
It can be obtained by and.

Next, the coordinates (x
The pixel value of ₀ , y ₀ ) is the coordinate (x _t ,
The pixel value of y _t ) is set (step 138).

The above-mentioned steps 134 to 138 are carried out for all the pixels of the texture-transformed image (step 140).

In the steps 134 to 140, the projective normalization inverse transformation is applied to the use area of the texture image T shown in FIG.
This is equivalent to creating a parsed texture image (assumed texture deformation image) T as shown in FIG.

After the parsed texture image T shown in FIG. 17 is created by the above processing, the parse designated area D in which the mask M is set in the original image G shown in FIG. The texture image T is combined using the image combining function of hardware to create a combined image as shown in FIG. 18 (step 142). afterwards,
The created combined image is output from the image combining device 14 to the image output device 16.

According to this embodiment described in detail above, the original image G
By setting the mask M having two window regions for the perspective designated region D of, it is possible to continuously synthesize images between the window regions, and therefore, for example, through a window seen through two windows. With such a landscape, it is possible to represent a continuous landscape between two windows with a natural sense of depth.

In the first embodiment, the projective normalized coordinates in the perspective designated area are calculated for each pixel in the perspective designated area and correspond to the projective normalized coordinates in the usage area of the texture image. In contrast to the procedure of obtaining absolute coordinates, according to the present embodiment, the projective normalization inverse transformation is applied to the texture image in advance, and the texture image is transformed into the same shape as the perspective designated area. Then, by performing a procedure of simply synthesizing it with the original image, an effect substantially equivalent to that of the first embodiment can be obtained.

Further, simple image composition without geometrical deformation like the composition process of the present embodiment can be easily realized by using hardware capable of high-speed image composition, and especially mask processing is also performed at the same time. It is possible to realize high-speed processing including masking by using the hardware that can be used.

Although the present invention has been specifically described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention.

For example, the shape of the window region formed in the mask region is arbitrary, and the number thereof is not particularly limited, and may be 3 or more.

Further, the pseudo perspective transformation applied to the texture image is not limited to the projective normalization inverse transformation, and the natural depth feeling is inferior. Good.

[0075]

As described above, according to the present invention, when synthesizing an original image and a texture image, no matter what shape the synthesis area of the original image is, the synthesis is performed without being affected by the shape. It is possible to produce a composite image having a natural sense of depth between the texture image pasted in the area and the other areas of the original image.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of projective normalization inverse transformation applied to the present invention.

FIG. 2 is a diagram showing a perspective designated area and a texture image to be pasted in the perspective designated area.

FIG. 3 is a diagram showing a texture image that has been subjected to normalized inverse transformation.

FIG. 4 is a diagram showing a texture image subjected to projective normalization inverse transformation.

FIG. 5 is a block diagram showing a schematic configuration of an image synthesizing apparatus according to the first embodiment of the present invention.

FIG. 6 is a flowchart for explaining the operation of the first embodiment.

FIG. 7 is another flowchart for explaining the operation of the first embodiment.

FIG. 8 is a diagram showing a method of designating a used area of a texture image.

FIG. 9 is a diagram showing a method of designating a perspective designating area of an original image.

FIG. 10 is a diagram showing an original image used in the example.

FIG. 11 is a diagram showing an original image in which a perspective designated area is designated.

FIG. 12 is a diagram showing an original image in which a mask is set in a perspective designated area.

FIG. 13 is a diagram showing a composite image in which an original image and a texture image are composited.

FIG. 14 is a flowchart for explaining the operation of the second embodiment.

FIG. 15 is another flowchart for explaining the operation of the second embodiment.

FIG. 16 is a diagram showing an original image in which a mask is set in a perspective designated area.

FIG. 17 is a diagram showing a texture image subjected to projective normalization inverse transformation.

FIG. 18 is a diagram showing a composite image in which an original image and a texture image are composited.

FIG. 19 is a diagram showing an original image in which a triangular window is formed.

FIG. 20 is a diagram of the window viewed from the front.

FIG. 21 is a diagram showing an example of a texture image.

FIG. 22 is a diagram showing an image in which a texture image obtained by coordinate conversion using the window as a perspective designated area is combined with an original image.

[Explanation of symbols]

 10 ... Image input device 12 ... Image area designation device 14 ... Image synthesis device 16 ... Image output device G ... Original image D ... Perspective designation area T ... Texture image M ... Mask

Claims (2)

[Claims] 1. A means for inputting an original image, a means for inputting a texture image, a means for designating a pasting area for pasting a texture image on the original image, and a texture image for the pasting area of the original image. In an image synthesizing apparatus including means for designating a usage area and image synthesizing means for synthesizing a usage area for a texture image in a pasting area of an original image, the image synthesizing means sets at least 1 in the pasting area of the original image. An image synthesizing apparatus having a function of setting a mask having one window area and a function of performing a pseudo perspective transformation on a usage area of a texture image. 2. The image synthesizing apparatus according to claim 1, wherein the pseudo perspective transformation function of the image synthesizing means is a function of performing projective normalization inverse transformation.