JPH1155567A - Key signal generator, its method, image synthesizer and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the key signal generator and its method in which areas to generate keys are properly divided on an image so as to conduct key generating processing in parallel to each area division and hence to reduce the processing time considerably. SOLUTION: A key generating area division section 3 configures an area to generate a key from an image with pluralities of division areas. A key processing selection section 5 selects a proper key generating processing procedure (program) for each of pluralities of division areas calculated by the key generating area division section 3. A key signal generating section 6 uses a proper key generating program selected by the key generating processing selection section 5 for each division area calculated by the key generating area division section 3 to generate a key signal keeping continuity.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a key signal generating apparatus and method for generating a key signal from an image composed of a foreground and a background, and a first image using the key signal generated by the key signal generating apparatus and method. The present invention relates to an image synthesizing apparatus for synthesizing a foreground and a background of a second image, and a recording medium recording a processing procedure relating to the key signal generation method.

[0002]

2. Description of the Related Art For example, in the production of movies and other images, various simulations of images (videos) are performed.
Alternatively, when giving a special effect to an image, a portion (object) of interest is extracted from the image, and the extracted portion is combined with another image. Extraction and synthesis from such an image are performed using a key signal.

The key signal includes a signal called a hard key and a signal called a soft key. For example, now
When the part of interest in the image used to generate the key signal is set to the foreground (foreground image) and the other part is set to the background (background image), the hard key is set to the target object as shown in FIG. Is a binary signal with α = 1 for the foreground and α = 0 for the background. Usually, the foreground portion is often expressed in white and the background portion is often expressed in black as shown in FIG.

On the other hand, as shown in FIG. 16, the soft key has a portion (0 ≦ α ≦ 1) where the value of the key changes smoothly from α = 0 to α = 1, so that the synthesis using the hard key is performed. In comparison, "blur" of the object can be expressed. The quality of combining a plurality of images depends largely on the condition of the soft key.

A key as described above is projected onto a plane, and the height of the key converted into a signal for each pixel is defined as a key signal. In order to perform good quality composition in image composition,
The key signal needs to change smoothly between pixels. For this reason, it is necessary to devise ways to make the keys that are the key signals continuous.

Conventionally, as a method of calculating a key signal required for image composition, there is a method disclosed in Japanese Patent Application Laid-Open No. Hei 5-236347. Japanese Patent Application Laid-Open No. Hei 5-23634
In the "soft key generation method" disclosed in Japanese Unexamined Patent Publication No. 7, a method of obtaining a hard key by shaving it with an elliptical cone as shown in FIG. 17 is proposed. According to this method, a motion vector near the boundary between the foreground and the background is calculated, and the major axis and the minor axis of the elliptical cone are determined in accordance with the motion vector.

On the other hand, in Japanese Patent Laid-Open Publication No. Hei 4-68763, an evaluation function for creating an arbitrary color belonging to the background from a plurality of areas (training areas) as a background area is created as a “cutout mask generation method”. A method of creating a mask (key signal) having a gradation for each pixel in a region (cutting area) to be cut out from an image is disclosed.

In this method, as shown in FIG.
When there are regions R ₁ and R ₂ of different colors along the boundary line of, an evaluation function is created for which region any pixel belongs to, and based on this evaluation function, Thus, it is possible to calculate a mask (key signal) of how much the pixel of the area to be cut out from the image belongs to the foreground.

[0009]

In the "soft key generation method" disclosed in Japanese Patent Laid-Open Publication No. Hei 5-236347, since the area for generating the key is not divided on the image, When it is desired to correct a part of the generated key signal, it is possible to correct the value of the key signal itself, but it is not possible to return to intermediate data in the middle of the portion requiring correction and perform the calculation again. In addition, a request to generate a key by a different method other than this method for each divided area cannot be satisfied.

In the "method of forming a cutout mask" disclosed in JP-A-4-68763, the region R
In the vicinity of ₁ and the boundary of the region R _2, there is the continuity of the mask created (key signal), for example, that the value of the key signal in the adjacent pixel is changing smoothly not guaranteed.

As described above, in the conventional method, a key is created by appropriately dividing an area for creating a key on an image.
In addition, there is no way to maintain the continuity of the created key as a whole, so there is no way to generate a key by a method suitable for each area, or to return to the intermediate data only in a part of the area where the key was generated and to repeat the key generation. Was.

The present invention has been made in view of the above circumstances, and by appropriately dividing an area for creating a key on an image, the key creation processing is performed in parallel for each of the divided areas. It is an object of the present invention to provide a key signal generation apparatus and method capable of greatly reducing the processing time.

It is another object of the present invention to provide a recording medium in which the above-described key signal generation method is programmed, so that a key signal whose processing time is greatly reduced can be easily generated.

Another object of the present invention is to provide an image synthesizing apparatus capable of greatly reducing the time required for key creation processing, and thus greatly reducing the time required for image synthesis processing.

[0015]

In order to solve the above-mentioned problems, a key signal generation apparatus according to the present invention comprises: a key generation area calculation means for forming a key generation area from an image by a plurality of divided areas; Key generation processing selection means for selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated by the key generation area calculation means; and key generation processing selection means for each divided area calculated by the key generation area calculation means And a key signal generating means for generating a key signal maintaining continuity by using an appropriate key generation processing procedure selected in (1).

Here, since the key generation processing selecting means selects an appropriate key generation processing procedure for each of the divided areas, it becomes possible to generate a key in a method suitable for each area and generate a key. Key generation can be performed again only in a part of the area.

Further, the key signal generating device according to the present invention comprises:
In order to solve the above-mentioned problems, a key generation area calculation unit configured to form a key generation area from an image by a plurality of divided areas, and an appropriate key generation process for each of the plurality of divided areas calculated by the key generation area calculation unit Key generation processing selection means for selecting a procedure, and a key signal for generating a key signal using an appropriate key generation processing procedure selected by the key generation processing selection means for each divided area calculated by the key generation area calculation means Generating means, and continuity maintaining means for maintaining continuity of the key signal generated by the key signal generating means.

Here, the continuity holding means can guarantee the continuity of the key signal generated by the key signal generating means, for example, that the value of the key signal changes smoothly in adjacent pixels.

Further, the key signal generating method according to the present invention comprises:
In order to solve the above problem, an area for generating a key from an image is calculated as a plurality of divided areas, an appropriate key generation processing procedure is selected for each of the plurality of divided areas, and using this key generation processing procedure, It is characterized in that a key signal that maintains continuity in each divided area is generated.

Further, the key signal generating method according to the present invention comprises:
In order to solve the above problem, an area for generating a key from an image is calculated as a plurality of divided areas, an appropriate key generation processing procedure is selected for each of the plurality of divided areas, and using this key generation processing procedure, A key signal is generated in each divided area, and continuity of the key signal is maintained.

According to another aspect of the present invention, there is provided an image synthesizing apparatus for receiving at least a first image including a foreground and a background and a second image including at least a background. An image synthesizing apparatus for synthesizing a foreground of the image and a background of the second image, a key generation area calculating means for forming a key generation area from the first image by a plurality of divided areas; A key generation process selection unit that selects an appropriate key generation process procedure for each of the plurality of divided regions calculated by the region calculation unit; and a key generation process selection unit that selects each of the divided regions calculated by the key generation region calculation unit. Key signal generation means for generating a continuous key signal using the appropriate key generation processing procedure,
A synthesizing unit that synthesizes a foreground of the first image and a background of the second image using the key signal generated by the key signal generating unit.

Here, since the key generation processing selecting means selects an appropriate key generation processing procedure for each of the divided areas, it becomes possible to generate a key in a method suitable for each area and generate a key. Key generation can be performed again only in a part of the area. For this reason, the operability of correction can be improved as an image synthesizing device, and the operation time can be reduced.

Further, in order to solve the above-mentioned problem, an image synthesizing apparatus according to the present invention receives at least a first image composed of a foreground and a background and a second image composed of at least a background, and An image synthesizing apparatus for synthesizing a foreground of the image and a background of the second image, a key generation area calculating means for forming a key generation area from the first image by a plurality of divided areas; A key generation process selection unit that selects an appropriate key generation process procedure for each of the plurality of divided regions calculated by the region calculation unit; and a key generation process selection unit that selects each of the divided regions calculated by the key generation region calculation unit. Key signal generating means for generating a key signal using the appropriate key generation processing procedure, continuity maintaining means for maintaining continuity of the key signal generated by the key signal generating means, and continuity maintaining means for maintaining the continuity of the key signal generated by the key signal generating means. By using the key signal continuity is maintained by means comprises a foreground of the first image, and a synthesizing means for synthesizing the background of the second image.

Here, the continuity holding means can guarantee the continuity of the key signal generated by the key signal generating means, for example, that the value of the key signal changes smoothly between adjacent pixels. The image synthesizing apparatus can realize high-quality image synthesizing processing.

Further, according to the recording medium of the present invention, a key generation area calculating step in which an area for generating a key from an image is composed of a plurality of divided areas, and a plurality of divided areas calculated in the key generation area calculating step are provided. A key generation processing selection step of selecting an appropriate key generation processing procedure, and for each divided area calculated in the key generation area calculation step, the appropriate key generation processing procedure selected in the key generation processing selection step is used. And a key signal generating step of generating a key signal maintaining the characteristics.

Further, according to the recording medium of the present invention, there is provided a key generation area calculating step in which an area for generating a key from an image is composed of a plurality of divided areas, and a plurality of divided areas calculated in the key generation area calculating step. A key generation processing selection step of selecting an appropriate key generation processing procedure, and a key signal using the appropriate key generation processing procedure selected in the key generation processing selection step for each divided area calculated in the key generation area calculation step. And a continuity maintaining step for maintaining continuity of the key signal generated in the key signal generating step.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a key signal generation apparatus and method, an image synthesizing apparatus and a recording medium according to the present invention will be described.

First, an embodiment of the key signal generating apparatus and method will be described. This embodiment is a key signal generation device that generates a key signal from an image composed of a foreground and a background, and operates each component by applying the above-described key signal generation method.

As shown in FIG. 1, the key signal generating apparatus includes a key generating area dividing section 3 serving as key generating area calculating means for forming a key generating area from the image by a plurality of divided areas; A key generation processing selecting unit 5 for selecting an appropriate key generation processing procedure (program) for each of the plurality of divided areas calculated by the generation area dividing unit 3, and a key generation processing selecting unit 5 for each of the divided areas calculated by the key generation area dividing unit 3. A key signal generation unit 6 that generates a key signal with continuity using an appropriate key generation program selected by the key generation processing selection unit 5.

The key generation area dividing section 3 divides a key generation area based on the contour detected by the contour detecting section 1.

Further, the key signal generating device is, for example,
The ROM 19 stores a first key generation processing procedure (program) 19 ₁ , a second key generation program 19 ₂ , a third key generation program 19 ₃ ... N-th key generation program 19 _n .

In the key signal generating apparatus, when the key signal generated by the key signal generating section 6 is further required for continuity, the switching is selected by the switcher 7 and the key signal is input. A continuity holding unit 8 for holding signal continuity, a monitor 17 for displaying a key signal via the key signal generation unit 6 or the continuity holding unit 8, and an operator switches the switcher 7 while watching the monitor 17. An input device 16 such as a mouse or a keyboard for instructing, and a switching control unit 9 for controlling switching of the switcher 7 by an instruction from the input device 16 are also provided.

Here, the contour detecting section 1, the key generating area dividing section 3, and the key signal generating section 6 are composed of a logical operation section (Arithmetic a).
nd logical Unit (ALU) 11. That is, the above-described units shown in FIG.
2 shows each functional block of the ALU 10 housed inside the PU 10.

In FIG. 2, the CPU 10 includes an ALU 11
, A register 12, and a controller 13.
After the input data supplied through the interface (I / F) 15 connected to the ALU 4 is temporarily stored in the RAM 18, the arithmetic operation by the ALU 11 is temporarily stored in the RAM 18 or the ROM 19, and then output.

The register 12 temporarily stores data used for calculation in the ALU 11, calculation results, and the like. The ALU 11 reads data held in the register 12, performs a predetermined operation on the data, and outputs the operation result to the register 12.

The controller 13 registers the register 12 and the AL
It controls U11 to perform calculations and the like stored in the program.

The RAM 18 is adapted to temporarily store data and programs used in the CPU 10.

The ROM 19 stores a program supplied from the CPU 10 in addition to storing a program for controlling the CPU 10 and outputs the data to the I / F 15. In this embodiment, as shown in FIG. 1, the ROM 19 stores a plurality of n key generation processing programs 19 ₁ , 19 ₂ , 19 _3.
.. 19 _n are stored.

For example, the operation of the key signal generation apparatus when the key generation processing selection unit 5 shown in FIG. 1 continuously selects the processing of generating a key using a curved surface controlled by parameters will be described with reference to FIG. This will be described with reference to the flowchart shown in FIG.

First, the contour detecting section 1 uses a Sobel operator filter (a filter for calculating the edge strength of an image) as shown in FIG. 4 to extract the contour of the input image. , The contour of the object A as indicated by the broken line is detected.

Next, in step S1, the key generation area dividing section 3 determines points P ₁ , P ₂ ,..., P which are considered to have a large curvature with respect to the contour curve, as shown in FIG. _n are calculated, and points P ₁ , P ₂ ,..., P _n at points P ₁ , P _{2 ,.} Calculate the line segments L ₁ , L ₂ ,..., L _n as the middle points. Then, the line segment L _1, L _{2, ...,} end point C ₁ of _{L n, C 2, ...,} C n and end points D _1, D _2, ..., respectively as group D _n, each Curves S ₁ , S formed by smoothly connecting adjacent end points of a group
_{2, ...,} S _n and the curve T _1, T _2, ..., to generate a T _n, thereby, the region area of the image _{R 1, R 2, ...,} R n and the region F, B is divided into (n + 2) areas.

Then, after initializing the number i of the region Ri to 1 in step S2 in FIG. 3, the key generation processing selecting unit 5 in step S3 considers continuity with the adjacent curved surface in the region Ri. , The key generation processing procedure is selected from the ROM 19.

Here, as described above, it is assumed that the key generation processing selecting unit 5 successively selects the process of generating a key using the curved surface controlled by the parameter.

Then, a program for generating a key using the curved surface controlled by the parameters is read from the ROM 19 to, for example, the RAM of the key signal generator 6, and the key signal generator 6 converts the key signal according to the program. , Through steps S5 and S6 until the region Rn is _reached .

Here, the process of generating a key using a curved surface controlled by parameters is the following key generation process. That is, as shown in FIG.
1, 2,. . . With n as the height of the curve S _i and 1 as the height of the curve T _i , a curved surface U _i that smoothly connects these two curves is generated. The height at each point of this curved surface is set to height 0
This is a process of generating a key signal by projecting (mapping) on the plane of (1) and determining a value for each pixel. This key generation processing is based on the technique of “key signal generation apparatus and method” disclosed by the present applicant in Japanese Patent Application Laid-Open No. 9-37153.

In the key signal generator, the key signal generated by the key signal generator 6 is displayed on the monitor 17, and the operator further maintains continuity between the respective areas as shown in step S7. When the input device 16 is operated as desired, the switcher 7 is switched by the switching control unit 9 to supply the respective key signals to the continuity holding unit 8, and post-processing of step S8 shown in FIG. 3 is performed. The post-processing here is a process for maintaining the continuity of the key signal generated by the key signal generation unit 6. Details will be described later.

Then, the process proceeds to a step S9, where key signals for the areas F and B are generated as required, and the flow is terminated.

Next, the operation of the embodiment in which the key generation processing selecting section 5 selects another key generation processing in consideration of the continuity and maintains the continuity of the key signal will be described below.

For example, in FIG. 5, if the region R ₂ does not require strict calculation, as shown in FIG. 7, the region R ₂ is divided into a triangle composed of points C ₂ , C ₃ , and D ₂ . As shown in FIG. 8, a triangle connecting points D ₂ , D ₃ and C ₃ is divided into triangles, and a line connecting points C ₂ and C ₃ has a height of 1 and a line connecting points D ₂ and D ₃ has a height of 1 as shown in FIG. Assuming that the height is 0, if this is mapped (projected) on a plane having a height of 0, a key signal continuous with an adjacent curved surface can be generated.

Then, the region R ₁ and the region R ₃ shown in FIG.
Are connected in a state where continuity is maintained to the key signal generated by the key generation processing shown in FIG.

This will be described with reference to the flowchart of FIG. First, in step S1, as described above, the image area is set to R ₁ , R ₂ , R ₃ R _n and, if necessary, F, B
Divided into And step S through step S2
In 3, the region R _1, to select the key generation process shown in FIG. 6, to generate a key signal at step S4.

[0052] Next, the the region R _2, if also okay with the key generation process without strict calculation, in step S4, and generates a key signal using the key generation process shown in FIG.

[0053] Next, the the region R _3, the key generation process selecting section 5 selects the key generation process shown in FIG. 6 in step S3, the key signal generating section 6 in step S4 generates a key signal.

Then, a key signal continuous with the adjacent curved surface as shown in FIG. 8 can be generated. However, in this case, a triangle composed of points C ₂ , C ₃ , D ₂ and points D ₂ , D ₃ ,
Since ridge occurs in the triangular border consisting of C _3, in order to obscure the boundary surfaces, it is necessary to devise so as to align also the direction of the plane.

[0055] Further, if the region hard key shape not including curve of R ₂ parts (foreground 1, background key for a binary zero) if may be represented by, as shown in FIG. 9 The foreground side and the background side of the boundary line connecting the points P ₂ and P ₃ of the region R ₂ may be filled with 1 and 0, respectively. At this time, the points C ₂ and D _2
Second , the points C ₃ and D ₃ move to the point P ₃ to recalculate the curved surfaces in the regions R ₁ and R ₃ , so that continuity with the adjacent curved surfaces can be maintained.

Up to this point, the key generation processing performed by the key generation program selected by the key generation processing selection unit 5 has been described.

Next, the operator looks at the image displayed on the monitor 17 and temporarily selects the region R _{1 as} shown in FIG.
In the case where the part of hits the boundary of the screen, the key generation program selected via the continuity holding unit 8
The key generated in the regions R ₁ may be clipped at the boundary line of the screen.

Similarly, if the regions R ₁ and R ₂ overlap because the outline is bent at a right angle, the keys generated in the regions R ₁ and R ₂ interfere with each other as shown in FIG. Therefore, for example, when projecting the height of the key onto a plane, the continuity of the curved surfaces of the key signals in the regions R ₁ and R ₂ can be maintained by prioritizing the minimum value and clipping each other.

In the case of the example shown in FIG. 11, since the contour is bent at a right angle, if the area is divided by the corner of the contour as shown in FIG. 11, there are two normal directions of the contour. , _Two points C ₂ and D ₂ are also present. In this case, keys formed from points C ₁ , C ₂ , D ₂ , D _{1 are} assigned to points C ₂ , D ₂
, And the key formed from the points C ₂ , C ₃ , D ₃ , D ₂ is extended in the direction of the points C ₂ , D ₂ . Write. Then, as shown in FIG. 11, a ridge line (shown by a thick line in FIG. 11) appears at a portion where both keys overlap.

Next, an example of a method of mixing the key signals so as to maintain continuity when the key signals are calculated by different methods in the regions R ₁ and R ₂ will be described.

As shown in FIG. 12, both the regions R ₁ and R ₂ project Bezier surfaces Q ₁ and Q ₂ controlled by the first-order parameter s and the third-order parameter t onto a plane to generate a key signal. Is calculated. Now, it is assumed that the regions R ₁ and R ₂ are given by the following equations.

Region R ₁ : Q ₁ (s, t) (0 ≦ s ≦ 1, 0 ≦ t ≦ 1) (1) Region R ₂ : Q ₂ (s, t) (0 ≦ s ≦ 1) , 0 ≦ t ≦ 1) (2) In this case, the curved surfaces Q ₁ ,
Continuous curved surface Q (s, t) by mixing Q ₂ to be able to generate, can be obtained a key signal which is continuous by further projecting the curved surface Q to plane.

Q (s, u) = (1-u) * Q ₁ (s, u) + u * Q ₂ (s, u) (3) where (0 ≦ s ≦ 1, 0 ≦ u ≦ 1).

The hatched portion in FIG. 12 indicates a region where the curved surface (s, u) is projected on the plane s = 0. With further generalize this, if in the process 1 on the left region R ₁ side, region R ₂
On the right side of the above, if a key is generated by the method 2, a key is generated for each of the regions R ₁ and R _{2 by two} methods of the methods 1 and 2 respectively, and for each pixel included in the regions R ₁ and R ₂ , By appropriately mixing the pixel values obtained by Method 1 and Method 2, a key that maintains continuity can be generated.

Here, as described above, the key generation processing described in FIGS. 10 to 12 is performed by the operator in the continuity holding unit 8 shown in FIG. This is a processing program. That is, it is executed as a post-process of step S8 which is performed when required by the determination in step S7 of the flowchart of FIG.

On the other hand, the key generation processing described in FIGS. 6 to 9 is selected in consideration of continuity by the key generation processing selection unit 5 shown in FIG. Is a key generation processing program to be executed.

The key signal generation device causes the key generation processing selection section 5 or the continuity holding section 8 to appropriately select the key generation processing program in accordance with the operation of the operator, and performs the processing shown in FIG. And the desired result can be obtained.

That is, the key signal generation device divides an area for generating a key by the key generation area dividing section 3 and selects an optimal key generation method for each of the divided areas.
Therefore, the processing can be parallelized for each of the divided areas, and as a result, the processing time can be greatly reduced.

Further, when it is desired to correct a part of the obtained key signal, only the area including the part to be corrected needs to be recalculated, so that the correction time can be greatly reduced.

Further, by selecting a method according to the situation of the divided area, the quality of the finally obtained key signal can be improved, and the time required to generate the key signal can be shortened. Further, by providing a method for maintaining the continuity of the generated key signals, even if a different key generation method is selected for each area, a continuous key signal can be finally obtained.

In the above embodiment, the method of dividing the region for generating the key is calculated based on the input image and the edge strength of the image. However, the operator directly inputs the method of dividing the region. The present invention is of course applicable to the case where a method or a method of dividing using another calculation method is used.

In the above embodiment, a method of generating a key for each divided area is a method of generating a three-dimensional curved surface or plane for each area and mapping this to a plane. Although the present invention has been used, it goes without saying that the present invention is also applicable to the case where another generation method is used to generate a key.

In the above embodiment, the method of maintaining the continuity of the key signal is a method of creating a key so as to maintain continuity with an adjacent area in each divided area. Even if you try to maintain continuity when finally combining keys created in the area,
Of course, the present invention is applicable.

Further, in the above embodiment, the method of maintaining the continuity of the key signal is used. However, in the area where the continuity of the key signal does not need to be maintained, the key is created without considering the continuity. Of course, it may be possible.

Next, an embodiment of the image synthesizing apparatus according to the present invention will be described below. FIG. 13 shows a configuration of an image synthesizing apparatus that synthesizes an image using the key signal generated by the key signal generating apparatus.

In FIG. 13, image providing devices 61 to 61 are provided.
Reference numeral 63 denotes a video tape player or a disk player such as a hard disk device for reproducing and outputting a predetermined image. Here, an image T1 (first image) whose foreground or background is F or B, respectively, reproduced by the image providing device 61 is supplied to the key signal generation device 64 and the mixer 65. This image T1 is a target for generating a key signal. The image T2 reproduced by the image providing device 62
The (second image) is supplied to the mixer 65, and the foreground F of the image T1 is combined with the image T2.

Further, from the image providing device 63, an image T3 obtained by photographing only the background B of the image T1 reproduced by the image providing device 61 is reproduced. That is,
For example, if the image T1 is a scene in which a person is walking in a certain landscape, an image T3 that captures the landscape (background B) with the person (foreground F) excluded is:
The image is reproduced from the image providing device 63.

The image T3 reproduced by the image providing device 63
Are supplied to a mixer 65.

The key signal generating device 64 has the same configuration as the key signal generating device shown in FIG. 1 and generates a continuous key signal α between the divided areas as described above. Output. The mixer 65 combines the images T1 and T2 input thereto using the key signal α from the key signal generation device 64. That is, the mixer 65
Extracts the foreground F from the image T1 and combines (fits) the foreground F with the image T2.

Next, the operation will be described. In the image providing devices 61 to 63, the images T1 to T3 are respectively reproduced and supplied to the mixer 65. Further, the image T1 reproduced by the image providing device 61 is also supplied to the key signal generating device 64.

In the key signal generation device 64, as described above, a continuous key signal α is generated between the divided areas.
Output to mixer 65. In the mixer 65, the key signal α
, The foreground F is extracted from the image T1 input thereto, and the foreground F is synthesized with the image T2, thereby generating an image (synthesized image) T4 obtained by synthesizing the foreground F with the image T2. Is done.

That is, the pixel value of the pixel at each position p forming the image T1 or the image T2 is represented by P (p) or Q (p), respectively, and each position forming the foreground or background of the image T1 is represented by Assuming that the pixel value of the pixel p is F (p) or B (p), and the pixel value of the pixel at each position p forming the composite image T4 is R (p), the following equation is established.

P (p) = αF (p) + (1−α) B (p) (4) R (p) = αF (p) + (1−α) Q (p) (5) Here, if αF (p) on the right side of the expression (5) is removed by the expression (4), the expression (5) becomes the following expression (6).

R (p) = P (p) − (1−α) B (p) + (1−α) Q (p) (6) In the mixer 65, according to the above equation (6), A composite image T4 is generated. That is, the pixel values P (p), Q
(P) and B (p) are supplied to the mixer 65 from each of the image providing devices 61 to 63, and the key signal α (p) is also supplied from the key signal generating device 64 to the mixer 65. The mixer 65 uses these pixel values and key signals to
(3) is performed, whereby the pixel value R (p) is calculated.
, Ie, an image T4 in which the foreground F is combined with the image T2.

The image T4 is supplied to, for example, a recording device (not shown) (for example, a video tape recorder or a writable disk recorder such as a hard disk device) and recorded.

The key signal generation device 64 generates the key signal α maintaining the continuity between the divided areas as described above. For this reason, since an optimal key generation method is selected for each of the divided areas, it is possible to parallelize the processing for each of the divided areas, and as a result, the processing time is greatly reduced, and The synthesis process is also greatly reduced.

The key signal generation method according to the present invention, that is, an area for generating a key from an image is calculated as a plurality of divided areas, and an appropriate key generation processing procedure is selected for each of the plurality of divided areas. By using a key generation processing procedure to generate a key signal that maintains continuity in each divided area, a program is recorded on a recording medium, and this program is read out as appropriate, so that it is easy to create Thus, a key signal that maintains continuity can be generated.

More specifically, this processing procedure includes a key generation area calculating step in which an area for generating a key from an image is composed of a plurality of divided areas, and a plurality of divided areas calculated in the key generation area calculating step. A key generation processing selection step of selecting an appropriate key generation processing procedure, and for each divided area calculated in the key generation area calculation step, the appropriate key generation processing procedure selected in the key generation processing selection step is used. And a key signal generation step of generating a key signal that maintains the characteristics.

Also, another key signal generating method according to the present invention,
That is, an area for generating a key from an image is calculated as a plurality of divided areas, an appropriate key generation processing procedure is selected for each of the plurality of divided areas, and using this key generation processing procedure,
By generating a key signal in each divided area and reading and executing the program from a recording medium recording a processing procedure (program) using a method of maintaining the continuity of the key signal, it is possible to easily execute the inter-divided area. Thus, a key signal that maintains continuity can be generated.

More specifically, this processing procedure includes a key generation area calculating step in which an area for generating a key from an image is composed of a plurality of divided areas, and a plurality of divided areas calculated in the key generation area calculating step. A key generation processing selection step of selecting an appropriate key generation processing procedure, and a key signal using the appropriate key generation processing procedure selected in the key generation processing selection step for each divided area calculated in the key generation area calculation step. And a continuity maintaining step for maintaining continuity of the key signal generated in the key signal generating step.

[0091]

According to the key signal generating apparatus and method of the present invention, by appropriately dividing an area for creating a key on an image, a key creating process is performed in parallel for each of the divided areas. And the processing time can be greatly reduced.

Further, the image synthesizing apparatus according to the present invention can greatly reduce the key creation processing time, and therefore can greatly reduce the image synthesis processing time.

Further, the recording medium according to the present invention provides a recording medium in which the above-described key signal generation method is programmed, so that a key signal whose processing time is greatly reduced can be easily generated.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a main part of an embodiment of a key signal generation device and method according to the present invention.

FIG. 2 is a block diagram of a key signal generation device including a main part for operating the functional blocks shown in FIG. 1;

FIG. 3 is a flowchart for explaining the operation of the key signal generation device.

FIG. 4 is a diagram showing a Sobel operator filter used in the key signal generation device.

FIG. 5 is a diagram for explaining an operation of a key generation area dividing unit included in the key signal generation device.

FIG. 6 is a diagram for explaining a process of generating a key using a curved surface controlled by parameters.

FIG. 7 is a diagram for explaining a key obtained by dividing an area into two triangles.

FIG. 8 is a diagram for explaining an operation of connecting the key shown in FIG. 7 to a key obtained from the key generation processing described in FIG. 6 and generating a key signal.

9 is a diagram for explaining a key generation process when the shape of the partial region R ₂ is may be represented by hard keys without the curve.

10 is a diagram for explaining the key generating process for clipping the key generated in the region R ₁ at the boundary line of the screen.

FIG. 11 is a diagram for explaining key generation processing when the contour is bent at a right angle.

FIG. 12 is a diagram for explaining a key generation process for mixing key signals in regions R ₁ and R _{2 by} different methods so as to maintain continuity.

FIG. 13 is a block diagram illustrating a configuration of an image composition device according to the present invention.

FIG. 14 is a diagram for explaining a hard key.

FIG. 15 is a diagram in which a foreground portion of a hard key is expressed in white and a background portion is expressed in black.

FIG. 16 is a diagram for explaining soft keys.

FIG. 17 is a diagram for explaining a “soft key generation method” disclosed in Japanese Patent Application Laid-Open No. 5-236347.

FIG. 18 is a diagram for explaining a “cutout mask creation method” disclosed in Japanese Patent Application Laid-Open No. 4-68763.

[Explanation of symbols]

3 key generation area division unit, 5 key generation processing selection unit, 6
Key signal generation unit, 8 continuity holding unit, 10 CPU, 1
1 Logical operation unit (ALU)

[Procedure amendment]

[Submission date] April 27, 1998

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0054

[Correction method] Change

[Correction contents]

Then, a key signal continuous with the adjacent curved surface as shown in FIG. 8 can be generated. However, in this case, a triangle composed of points C ₂ , C ₃ , D ₂ and points D ₂ , D ₃ ,
Since a ridgeline is formed at the boundary of the triangle made of C _3, it is necessary to devise the direction of the surface at the boundary of the surface so as to make the boundary of the surface inconspicuous.

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0062

[Correction method] Change

[Correction contents]

Region R1: Q ₁ (s, t) (0 ≦ s ≦ 1, 0 ≦ t ≦ 1) (1) Region R2: Q ₂ (s, t) (0 ≦ s ≦ 1, 0) ≦ t ≦ 1) (2) In this case, the curved surface Q ₁ ,
Curved Q (s, u) continuous by mixing Q ₂ to be able to generate, can be obtained a key signal which is continuous by further projecting the curved surface Q to plane.

[Procedure amendment 3]

[Document name to be amended] Drawing

[Correction target item name] Figure 3

[Correction method] Change

[Correction contents]

FIG. 3

Claims (20)

[Claims] 1. A key signal generation device for generating a key signal from an image composed of a foreground and a background, comprising: a key generation area calculation unit configured to form a key generation area from the image by a plurality of divided areas; A key generation processing selecting means for selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated by the calculating means; and a key generation processing selecting means for each divided area calculated by the key generation area calculating means. A key signal generation unit that generates a key signal with continuity using an appropriate key generation processing procedure. 2. The key generation processing selecting means selects a key generation processing procedure for generating a key using a curved surface controlled by a predetermined parameter in all of the divided areas, and the key signal generating means selects the curved surface. The key signal generation device according to claim 1, wherein the key signal is generated while projecting the key signal by projecting the key signal on a plane. 3. The key generation processing selecting means selects a key generation processing procedure for generating a key using a curved surface controlled by a predetermined parameter in a specific divided area, and generates a key. In other divided areas adjacent to the area, select a different key generation processing procedure to generate a key,
2. The key signal generating apparatus according to claim 1, wherein said key signal generating means generates a key signal maintaining continuity by using these keys. 4. A key signal generation apparatus for generating a key signal from an image composed of a foreground and a background, comprising: a key generation area calculation unit configured to form a key generation area from the image by a plurality of divided areas; A key generation processing selecting means for selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated by the calculating means; and a key generation processing selecting means for each of the divided areas calculated by the key generation area calculating means. Key signal generation comprising: key signal generation means for generating a key signal using an appropriate key generation processing procedure; and continuity holding means for maintaining continuity of the key signal generated by the key signal generation means. apparatus. 5. The continuity holding means appropriately mixes a key generated for each divided area calculated by the key generation area calculating means with a key of an adjacent divided area. 5. The key signal generation device according to 4. 6. The method according to claim 4, wherein said continuity holding means clips overlapping parts when combining keys generated for each of the divided areas calculated by said key generation area calculating means. The key signal generation device as described in the above. 7. The method according to claim 4, wherein the continuity holding means shares a key boundary generated for each divided area calculated by the key generation area calculating means with each other.
The key signal generation device as described in the above. 8. The key signal generation device according to claim 4, wherein said continuity holding means uses two curves sandwiching a foreground contour. 9. A key signal generation method for generating a key signal from an image composed of a foreground and a background, wherein an area for generating a key from the image is calculated as a plurality of divided areas, and an appropriate key is set for each of the plurality of divided areas. A key signal generation method comprising: selecting a generation processing procedure; and using the key generation processing procedure, generating a key signal that maintains continuity in each divided area. 10. A key generation procedure for generating a key using a curved surface controlled by a predetermined parameter in all of the divided regions, and projecting the curved surface onto a plane to convert a key signal into a continuity signal. 10. The key signal generation method according to claim 9, wherein the key signal is generated while maintaining the following. 11. A key generation processing procedure for generating a key using a curved surface controlled by a predetermined parameter in a specific divided area to generate a key, and the other divided area adjacent to the specific divided area is generated. 10. The key signal generation method according to claim 9, wherein different key generation processing procedures are selected to generate keys, and a key signal with continuity is generated using these keys. 12. A key signal generating method for generating a key signal from an image composed of a foreground and a background, wherein an area for generating a key from the image is calculated as a plurality of divided areas, and an appropriate key is set for each of the plurality of divided areas. A key signal generation method comprising: selecting a generation processing procedure; generating a key signal in each divided area using the key generation processing procedure; and maintaining continuity of the key signal. 13. A key generated for each of the divided areas,
13. The key signal generating method according to claim 12, wherein the continuity of the key signal is maintained by appropriately mixing the keys with the keys of the adjacent divided areas. 14. The key signal generation method according to claim 12, wherein, when synthesizing the keys generated for each of the divided areas, the continuity of the key signal is maintained by clipping overlapping parts to each other. . 15. The key signal generation method according to claim 12, wherein the continuity of the key signal is maintained by sharing a boundary of the key generated for each of the divided areas with each other. 16. The key signal generation method according to claim 12, wherein the continuity of the key signal is maintained by using two curves sandwiching a foreground contour. 17. A first system comprising at least a foreground and a background
And a second image including at least a background, and synthesizing a foreground of the first image and a background of the second image, wherein a key is generated from the first image. Key generation area calculation means for configuring an area to be formed by a plurality of divided areas; key generation processing selection means for selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated by the key generation area calculation means; A key signal generation unit that generates a key signal with continuity by using an appropriate key generation processing procedure selected by the key generation processing selection unit for each divided region calculated by the generation region calculation unit; An image synthesizing apparatus, comprising: synthesizing means for synthesizing a foreground of the first image and a background of the second image using the key signal generated by the generating means. 18. A first system comprising at least a foreground and a background
And a second image including at least a background, and synthesizing a foreground of the first image and a background of the second image, wherein a key is generated from the first image. Key generation area calculation means for configuring an area to be formed by a plurality of divided areas; key generation processing selection means for selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated by the key generation area calculation means; Key signal generation means for generating a key signal using an appropriate key generation processing procedure selected by the key generation processing selection means for each divided area calculated by the generation area calculation means; and a key generated by the key signal generation means. Combining the foreground of the first image and the background of the second image using continuity holding means for maintaining signal continuity, and the key signal having continuity maintained by the continuity holding means You Image synthesizing apparatus characterized by comprising a combining means. 19. A recording medium recording a processing procedure for generating a key signal from an image composed of a foreground and a background, comprising: a key generation area calculation step of forming a key generation area from the image by a plurality of divided areas; A key generation processing selecting step of selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated in the key generation area calculation step; and the key generation processing for each of the divided areas calculated in the key generation area calculation step. A recording medium characterized by recording a processing procedure including a key signal generation step of generating a key signal maintaining continuity using an appropriate key generation processing procedure selected in the selection step. 20. A recording medium recording a processing procedure for generating a key signal from an image consisting of a foreground and a background, comprising: a key generation area calculation step of forming a key generation area from the image by a plurality of divided areas; A key generation processing selecting step of selecting an appropriate key generation processing procedure for each of the plurality of divided areas calculated in the key generation area calculation step; and the key generation processing for each of the divided areas calculated in the key generation area calculation step. A key signal generation step of generating a key signal using an appropriate key generation processing procedure selected in the selection step, and a continuity maintaining step of maintaining continuity of the key signal generated in the key signal generation step. A recording medium characterized by being recorded.