JP3957479B2 - Video signal editing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a video signal editing apparatus that synthesizes and outputs a video signal captured by an image input apparatus with a superimposed image.
[0002]
[Prior art]
For example, in a sticker manufacturing apparatus installed in a game center, a supermarket, or the like, a composite image obtained by combining a still image superimposed image so as to cover the face video signal input from the image input device is output as a sticker. be able to.
[0003]
Also, in recent mobile phone devices with cameras, videophone systems, etc., the image of the sender's face captured by the image input device is transmitted to the other party, and the sender's face is displayed on the display of the other party's device. it can. Furthermore, a videophone or the like is known that can transmit a face image to be transmitted after masking with a still image.
[0004]
FIG. 9 is a block diagram showing a conventional videophone described in Japanese Patent Laid-Open No. 5-64180 as an example of a videophone device that transmits an image after masking with a still image.
In the videophone device 500 of FIG. 9, the video signal input from the image input device 501 is once captured in the frame memory 503 via the image processing unit 502. An example of the image input device 501 is an imaging unit.
[0005]
Next, the image processing unit 502 combines (overlays) the video signal read from the frame memory 503 so as to cover the masking image read from the binary image memory 504, and then transmits the combined image to the image output device 505. Is output. Further, as the image output device 505, for example, a transmission unit can be cited.
[0006]
FIG. 10 is a diagram showing an output image combined with each input image in the videophone 500 of FIG.
FIG. 10A shows a captured image 550 taken by the imaging unit of the image input device 501. For example, in a videophone or the like, the appearance (face) 551 of the sender is shown in the background 552. .
[0007]
FIG. 10B shows a masking image 560 that is a binary image of a still image, and has a transmission part 561 in the masking part 562.
[0008]
FIG. 10C shows an output image obtained by combining (overlaying) the image of FIG. 10A and overlaying FIG. 10B. The background 552 is masked by the masking unit 562 and transmitted. In the part 561, the image of the sender (face) 551 appears.
[0009]
As described above, in a conventional videophone device or the like, the image input device is operated so that the sender (face) 551 enters the position of the transmission unit 561 provided in the masking unit 562 of the still image, The background 552 could be masked by the sender moving according to the position of the transmission part 561.
[0010]
For example, when a sender of a conventional videophone device wants to transmit a person's face but does not want to transmit a background, the sender uses a masking unit to set the imaging position of the face in advance. Was fit within the area. That is, in this case, the sender needs to match the background and the masking part or match the face and the transmissive part by his / her manual operation.
[0011]
Also, for example, when the sender does not want to transmit the background and the imaging position of the face cannot be adjusted within the area of the transmission part, transmission of all images is stopped and only audio is transmitted. There was only.
[0012]
[Problems to be solved by the invention]
As described above, in a conventional videophone device or the like, when the background portion of the captured image is not to be transmitted, as described above, the sender masks the background by matching the face and the transparent portion. Alternatively, since the transmission of the image has only been canceled, for example, when the position of the person (sender) in the input image is moving, the sender can manually change the position of the face to the position of the transmission part. We had to carry out the troublesome work of matching. In addition, when it is difficult to match the position of the face and the position of the transmission part, there is a problem that it is necessary to give up image transmission.
[0013]
In addition, when the sender is moving, the position of the sender's face and the position of the image input device also fluctuate, so the size of the person (face) in the input image changes, and the still image masking unit When the face is small, the surrounding background is reflected in the transmission part, and when the face is large, only a part of the face is reflected and the entire face is not projected.
[0014]
The present invention has been made to solve the above-mentioned problems, and is a video signal editing that can automatically detect only a human face as a specific area from an input image and mask and output other areas. An object is to provide an apparatus.
[0015]
[Means for Solving the Problems]
In order to achieve the above object, the video signal editing apparatus according to claim 1 is a specification that detects a specific area from a video signal input from an image input device and outputs position data in a frame of the specific area. Based on the position data, the position of the superimposed image is changed based on the position data, a region detection unit, a superimposed image storage unit that stores a superimposed image that is synthesized so as to cover at least a part of the image input from the image input device A superimposed image editing unit, and an image synthesis unit that synthesizes the video signal input from the image input device and the superimposed image, and the specific region detection unit detects a size of the specific region and outputs size data. And the superimposed image editing unit detects a dimensional difference by comparing the dimensional data with the superimposed image, and changes the magnification of the superimposed image based on the dimensional difference. The superimposed image stored in the superimposed image storage unit is a moving image, and the superimposed image editing unit converts the superimposed image of the moving image that changes over time into the position data and the dimension data. It is characterized by changing the position and magnification in an adaptive manner.
[0017]
In the video signal editing apparatus according to claim 2, the superimposed image stored in the superimposed image storage unit has a background part that can be freely transmitted or transmitted, and the superimposed image editing unit is controlled from the outside. It has a means for changing the transparency of the background portion according to the signal.
[0019]
The video signal editing apparatus according to claim 3, superimposed image editing unit, transmitted through the transmission of the background video in the superimposed image with the lapse of time, gradually to change the transmission from a non-transparent, or from non-transparent It is characterized by gradually changing to.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a video signal editing apparatus according to Embodiment 1 of the present invention.
In the figure, reference numeral 1 denotes an image input device that includes an imaging device such as a camera and outputs an input image as a digital video signal. Reference numeral 2 denotes a video signal editing apparatus that can detect the position of a face in a video signal from the video signal, change the position of the transmission part of the masking image in accordance with the face position, and synthesize (overlay) the video signal. . An image output device 3 displays an image of the synthesized video signal. When the video signal editing apparatus 2 according to the present embodiment is used in, for example, a videophone apparatus, the synthesized video signal is first input to an encoding apparatus (not shown) and becomes a transmission code. Sent to the videophone device. The encoding device is, for example, MPEG4, H.264, or the like. The video signal is encoded using an image encoding method such as H.263.
[0021]
Further, in the video signal editing device 2, 11 detects a sender's face or the like as a specific region from the input image data P 1 that is a video signal input from the image input device 1, and data relating to the detected region (region data) : AR) is a specific area detector. Reference numeral 12 denotes a superimposed image storage unit that stores a still image masked image or a still image superimposed image to modify an image such as a sender's face and outputs it as superimposed image data P2. Reference numeral 13 denotes a superimposed image editing unit that changes the display position of the superimposed image data P2 in the image frame based on the area data AR. Reference numeral 14 denotes an image composition unit that synthesizes the superimposed image data P2 so as to cover (overlay) the input image data P1, and outputs the composite image data P3.
[0022]
FIG. 2 is a block diagram showing a partial configuration of the video signal editing apparatus 2 according to the present embodiment.
The specific region detection unit 11 determines a specific region boundary detection unit 21 that detects a boundary of a specific region such as a face from the input image data P1 to be input, and determines the coordinate position of the detected boundary in the image frame. And a coordinate position determination unit 22 that outputs position data AR-P. As a face detection method, for example, the face region may be detected by a method such as skin color region detection or motion detection using color difference information, luminance difference information, shape determination, and the like. The coordinate position may be determined, for example, by determining the coordinate positions of all the contours, or obtaining the center value from the maximum and minimum values of the face contour on the X and Y axes.
[0023]
The superimposed image editing unit 13 shifts (edits) the display position of the input superimposed image data P2 in the image frame so as to match the position data AR-P and then outputs the superimposed image data MP2. Part 34 is provided.
[0024]
The operation of the video signal editing apparatus according to this embodiment will be described below.
FIG. 3 is a diagram showing input image data and output image data of the video signal editing apparatus 2 shown in FIG.
FIG. 3A is a captured image by the input image data P <b> 1 input from the image input device 1. In the captured image 100, the sender's face 110 and its background 112 are displayed. The specific area detected in the present embodiment is the face 110 of the sender, and the contour 110a of the face 110 is detected by, for example, color difference information, luminance difference information, or shape determination.
[0025]
FIG. 3B is a superimposed image such as a masking image stored in the superimposed image storage unit (image memory) 12 of FIG. The superimposed image 200 includes a face transparent area 201 for displaying the sender's face even after composition, a face surrounding non-transparent area 202 for masking or modifying the periphery of the face, and a background for displaying the background after composition. And a background transmissive region 203.
[0026]
FIG. 3C shows an output video image by the synthesized image data P3 output from the image synthesis unit 14 of the video signal editing device 2. In the output video 300, the non-transparent area 202 for the periphery of the face shifted in accordance with the position of the face 110 of the sender is displayed, or the face 110 is displayed through the transparent area 201 for the face, Further, the background 112 is displayed through the transparent area 203 for background.
[0027]
FIG. 4 is a diagram illustrating a coordinate determination operation by the coordinate position determination unit 22.
The image portion of the sender's face 110 is separated from the image portion of the scenery 112 by the face outline (boundary line) 110 a detected by the specific region boundary detection unit 21. The distance in each direction of the boundary line 110a from the upper left end when the sub-scanning direction of the scanning line is taken on the X axis and the main scanning direction is taken on the Y axis is measured.
[0028]
A point where the x-axis value on the boundary line 110a is the minimum value is A, and the coordinates of the point A are (X1, Y1). Also, let B be the point where the value of the x-axis is the maximum value, and let the coordinates of that point B be (X2, Y2). Similarly, the point where the y-axis value is the minimum value is C, the coordinate of the point C is (X3, Y3), the point where the y-axis value is the maximum value is D, and the coordinates of the point D are the same. Is (X4, Y4).
[0029]
In the image portion of the sender's face 110, the range on the x-axis is from X1 to X2, and the range on the y-axis is from Y3 to Y4. For example, as the center point of the image portion of the sender's face 110, the coordinate point on the x-axis may be (X2-X1) / 2, and the coordinate point on the y-axis may be (Y4-Y3) / 2. .
[0030]
When the face transparent area 201 in FIG. 3B is circular or elliptical, the center point of the face transparent area 201 is ((X2-X1) / 2, (Y4-Y3) / 2). By shifting the display position of the superimposed image 200 so that the image shown in FIG. 3C can be obtained.
[0031]
As described above, in this embodiment, the display position of the masking image (overlay image) can be automatically shifted according to the image position of the sender's face, so that the sender can adjust the position himself. This eliminates the need to perform complicated operations. In addition, since the face position and the transmissive part position can be automatically matched, the situation of giving up image transmission can be avoided.
[0032]
Embodiment 2. FIG.
In the first embodiment described above, the position data of the face is detected by the specific area detection unit 11 and the display position of the masking image is automatically shifted by the superimposed image editing unit 13. If it is, not only the position but also the face size may change. In the second embodiment described below, a case will be described in which size data is detected in addition to face position data, and the display position and size of the masking image are automatically changed.
[0033]
FIG. 5 is a block diagram showing a partial configuration of the video signal editing apparatus 2 according to the present embodiment.
The specific area detecting unit 11a is based on position data AR-P output from the coordinate position determining unit 22 in addition to the specific area boundary detecting unit 21 and the coordinate position determining unit 22 having the same functions as those in the first embodiment. An area dimension measuring unit 23 that measures area dimensions and outputs dimension data AR-L. As an area dimension measuring method, for example, the horizontal dimension of the face is measured by the calculation of X2-X1 shown in FIG. 4, the vertical dimension of the face is measured by the calculation of Y4-Y3, and the result is obtained. What is necessary is just to output as dimension data AR-L.
[0034]
The superimposed image editing unit 13a includes a region comparison unit 32 that compares the input superimposed image data P2 with the dimension data AR-L, and a region comparison before the coordinate position shift unit 34 having the same function as in the first embodiment. An image magnification changing unit 33 that receives the comparison result of the unit 32 and changes the display magnification of the superimposed image data P2.
[0035]
In the coordinate position shift unit 34 of the present embodiment, the display position of the superimposed image data P2 whose display magnification has been changed by the image magnification change unit 33 is shifted by the position data AR-P.
The operation of the video signal editing apparatus according to this embodiment will be described below.
[0036]
FIG. 6 is a diagram showing input image data and output image data of the video signal editing apparatus 2 according to the present embodiment.
FIG. 6A is a captured image by the input image data P <b> 1 input from the image input device 1. The sender's face 110 in the captured image 120 in the present embodiment is displayed larger than that in the first embodiment. Other display contents are the same as those in the first embodiment.
[0037]
FIG. 6B is a superimposed image such as a masking image stored in the superimposed image storage unit (image memory) 12 of FIG. 1 and is the same as that of the first embodiment.
FIG. 6C shows an output video image by the synthesized image data P3 output from the image synthesis unit 14 of the video signal editing apparatus 2.
[0038]
In the output video 310, the non-transparent area 302 for the face periphery whose position is shifted after being enlarged in accordance with the size of the face 130 of the sender is displayed, and the face 130 is displayed through the transparent area for the face. In addition, the background 132 is displayed through the enlarged background transparent area 203.
[0039]
As described above, in this embodiment, even when the face size in the captured image changes as shown in FIG. 6A, the same masking image (overlay image) as in the case where the face size does not change is used. The display position and size of the masking image can be automatically enlarged or reduced in accordance with the face area.
[0040]
Embodiment 3 FIG.
In the first and second embodiments, the background transparent area 203 is prepared in addition to the face transparent area 201 in the overlay image. However, the background may be made non-transparent.
[0041]
In the first and second embodiments described above, the transparent region 203 is also provided in the background of the masking image so that a part of the background of the captured image can be seen. For example, it is desired that the background of the captured image is not visible at all. In some cases. Therefore, in the third embodiment described below, a case where the background of the captured image is completely hidden will be described.
[0042]
FIG. 7 is a block diagram showing a partial configuration of the video signal editing apparatus 2 according to the present embodiment.
The specific area detection unit 11 b includes a specific area boundary detection unit 21, a coordinate position determination unit 22, and a region dimension measurement unit 23 having the same functions as those in the second embodiment.
In the superimposed image editing unit 13b, in addition to the region comparison unit 32, the image magnification changing unit 33, and the coordinate position shift unit 34 having the same functions as those in the second embodiment, the superimposed image data P2 is received by an external control signal. A background transparency adjusting unit 31 that can change the transparency of the background.
[0043]
In the area comparison unit 32 according to the present embodiment, the superimposed image data P2 whose background transparency is adjusted by the background transparency adjustment unit 31 is compared with the dimension data AR-L, and the magnification is changed by the image magnification change unit 33. To do.
[0044]
In the coordinate position shift unit 34 of the present embodiment, the display position of the superimposed image data P2 in which the background transparency is adjusted by the background transparency adjusting unit 31 and the display magnification is changed by the image magnification changing unit 33, Shift with position data AR-P.
[0045]
The operation of the video signal editing apparatus according to this embodiment will be described below.
FIG. 8 is a diagram showing input image data and output image data of the video signal editing apparatus 2 of the present embodiment.
FIG. 8A shows a captured image by input image data P1 input from the image input apparatus 1, and is the same as that in the second embodiment.
[0046]
FIG. 8B is a superimposed image such as a masking image stored in the superimposed image storage unit (image memory) 12 of FIG. 1. The position and shape are the same as those in the first embodiment, but the background is not. It is a transparent image.
[0047]
FIG. 8C shows an output video image by the synthesized image data P3 output from the image synthesis unit 14 of the video signal editing device 2. In the output video 320, the non-transparent area 302 for the face periphery whose position is shifted after being enlarged in accordance with the size of the face 130 of the sender is displayed, and the face 130 is displayed through the transparent area for the face. The point is the same as that of the second embodiment, but the enlarged background transparent region 314 is displayed.
[0048]
As described above, in this embodiment, even when the face size in the captured image changes as shown in FIG. 8A, the same masking image (overlay image) as in the case where the face does not change is used. The background portion can be automatically masked according to the display position and size of the masking image.
[0049]
Embodiment 4 FIG.
In the above-described third embodiment, the case where the masking image (overlay image) is a still image has been described. 12 can be stored.
[0050]
In this case, when the position and size of the transparent area of the face portion in the animation image whose position and size change with the passage of time are known, the image area change shown in FIG. The coordinate position and dimension position of the masking image are controlled by the unit 33 and the coordinate position shift unit 34.
[0051]
Further, when the position and size of the transparent region of the face part in the animation image are unknown, for example, the specific region boundary detection unit 21, the coordinate position determination unit 22, and the region size measurement unit 23 in the specific region detection unit 11b By providing a similar circuit unit at the input stage of the superimposed image editing unit 13b, it is possible to know the position and size of the transmission region of the face portion. The subsequent control is the same as when the position and dimensions are known.
[0052]
In this way, in this embodiment, an animation can be used as a masking image (overlay image), and the captured image is automatically displayed in the transparent region of the face according to the display position and size of the masking image of the animation. You can put your face or mask the background.
[0053]
Embodiment 5 FIG.
In the fourth embodiment, the case where the masking image is either transparent or non-transparent has been described. At that time, for example, the background transparency is adjusted by using a fade-in / fade-out method. By controlling the unit 31, it may be configured to switch from the transmitted background to the masking image as time passes.
[0054]
In that case, for example, when the sender reaches a place where the background is desired to be hidden while moving, the superimposed image editing unit 13 gradually changes the transparency of the background of the superimposed image from transparent to non-transparent over time. By (fade out), the background can be masked with less discomfort due to sudden switching. Similarly, when the sender approaches a place where the background does not need to be hidden while moving, the transparency of the background of the superimposed image is gradually changed from non-transparent to transparent (fade in) as time passes. The background can be displayed with less discomfort to the other party.
[0055]
In this way, in this embodiment, for example, even if the sender is moving, the background can be hidden or displayed depending on the location, and the background can be faded in or faded out. As a result, it is possible to reduce the situation in which the person who sees the image feels uncomfortable.
[0056]
In each of the above embodiments, the face is detected as the specific area detected from the input image data. However, the present invention is not limited to this, and any area that can be specified from the input image is used. Can be applied. For example, the hand or fingertip may be used as a specific image and the other may be masked, or the background may be masked using an animal or a moving body as a specific area.
[0057]
【The invention's effect】
As described above, in the present invention described in claim 1, the display position of the masking image (overlay image) can be automatically shifted in accordance with the image position of the sender's face. Since it is not necessary to perform position adjustment, complicated operations can be omitted, and the position of the face and the position of the transmission part can be automatically matched, so that the situation of giving up image transmission can be avoided. .
[0058]
According to the first aspect of the present invention, even when the size of the face in the captured image changes, the masking image is used by using the same masking image (overlay image) as that in the case where the size does not change. Can be automatically enlarged or reduced in accordance with the face area.
[0059]
Further, in the present invention described in claim 2 , in addition to the fact that the same masking image (overlay image) can be used even when the face size in the captured image changes, the masking image can be used. The background portion can be automatically masked according to the display position and size.
[0060]
According to the first aspect of the present invention, an animation can be used as a masking image (overlay image), and an image is automatically captured in the transparent region of the face according to the display position and size of the masking image of the animation. You can put a face in the image or mask the background.
[0061]
Further, in the present invention described in claim 3 , even when the sender is moving, the background can be hidden or the background can be displayed depending on the location, and the background can be faded in or faded out. As a result, it is possible to reduce the situation in which the person who sees the image feels uncomfortable.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a video signal editing apparatus according to Embodiment 1 of the present invention.
2 is a block diagram showing a partial configuration of the inside of the video signal editing apparatus of FIG. 1;
FIGS. 3A to 3C are diagrams showing input image data and output image data of the video signal editing apparatus shown in FIG.
FIG. 4 is a diagram illustrating a coordinate determination operation by a coordinate position determination unit.
FIG. 5 is a block diagram showing a configuration of a part of a video signal editing apparatus according to a second embodiment of the present invention.
6A to 6C are diagrams showing input image data and output image data of the video signal editing apparatus of FIG.
FIG. 7 is a block diagram showing a partial configuration of a video signal editing apparatus according to Embodiment 3 of the present invention.
8A to 8C are diagrams showing input image data and output image data of the video signal editing apparatus of FIG.
FIG. 9 is a block diagram showing a conventional videophone.
FIGS. 10A to 10C are diagrams showing output images synthesized with respective input images in the videophone of FIG. 9;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image input device (imaging device), 2 Video signal editing device, 3 Image output device, 11 Specific area (face) detection part, 12 Superimposed image memory | storage part, 13 Superimposed image editing part, 14 Image composition part, 21 Specific area boundary A detection unit, 22 a coordinate position determination unit, 23 an area size measurement unit, 31 a background transparency adjustment unit, 32 an area comparison unit, 33 an image magnification change unit, and 34 a coordinate position shift unit.

Claims (3)

A specific area detection unit that detects a specific area from a video signal input from an image input device and outputs position data in a frame of the specific area;
A superimposed image storage unit that stores a superimposed image synthesized so as to cover at least a part of an image input from the image input device;
A superimposed image editing unit that changes the position of the superimposed image in a frame based on the position data;
An image synthesis unit that synthesizes a video signal input from an image input device and the superimposed image ;
The specific area detection unit includes means for detecting a dimension of the specific area and outputting dimension data;
The superimposed image editing unit has a means for detecting a dimensional difference by comparing the dimensional data with the superimposed image, and changing a magnification of the superimposed image based on the dimensional difference,
The superimposed image stored in the superimposed image storage unit is a moving image,
The video signal editing apparatus, wherein the superimposed image editing unit changes a position and a magnification of a superimposed image of a moving image that changes over time according to the position data and the dimension data . The superimposed image stored in the superimposed image storage unit has a background part that can be transparent or non-transparent,
The video signal editing apparatus according to claim 1, wherein the superimposed image editing unit includes means for changing the transparency of the background portion by a control signal from the outside. Prior Symbol superimposed image editing unit, the transparency of the background video superimposed image with the lapse of time, or gradually changed in the transmission from a non-transparent, or claim 2, characterized by gradually changing the transmission from the non-transparent The video signal editing apparatus described in 1.

Images