JP4427140B2 - Object video display device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an object video display device capable of displaying a plurality of object videos encoded by, for example, the MPEG method, and in particular, detects an area that can be pasted with another video in an object video, and matches the area The present invention relates to a technique for selecting an image and pasting and displaying it on the area.
[0002]
[Prior art]
There is a technique in which a region to be corrected is set for an image to be displayed, and another material data is inserted into the region to create another new image as a whole. For example, Japanese Patent Application Laid-Open No. 8-287225 discloses a technique for displaying a three-dimensional image of a house by replacing a part of the wall with another new material data. In this conventional example, when the wall is oblique in the image, the material data is also changed in accordance with the original wall area. When the material data is inserted, it is assumed that the area to be inserted is previously selected on the target image manually by a mouse or the like. In other words, the vertex of the wall is set with the mouse, a polygon is generated, the material data is deformed according to the polygon, and the material data is pasted to the corresponding area.
[0003]
FIG. 6 is an explanatory diagram showing an example in which another video is pasted on the object video. FIG. 6 shows an example in which a logo object video 104 such as a number, a person's name, or a CM is attached to the object video 102 of the person who is the target video or the target subject. The person object video 102 is used for general purposes, and the object video 102 can be used for various purposes by pasting a logo object video 104 (for example, the name of this person) on a partial area thereof. In addition, a mask object video 110 is pasted on the object video 108 of another person in order to hide the face area. By concealing the face of the person object video 108, it is difficult to know who the object video is, and this helps protect privacy. Reference numeral 112 denotes a decorative frame object image. By pasting the character object video 114 on the decorative frame object video 112, a plurality of general-purpose decorative frame object videos 112 are prepared, and by changing the contents of the character object video 114 as appropriate, there are various situations in the scene. Can be realized.
[0004]
[Problems to be solved by the invention]
In the technique disclosed in the above-mentioned Japanese Patent Laid-Open No. 8-287225, an area where a material image is to be pasted must be manually designated with a mouse or the like with respect to the original house image. For this reason, the original image is limited to a still image because it requires human labor. Further, conventionally, the position of the logo object video 104 pasted on the object video 102 in FIG. 6, the position of the mask object video 110 pasted on the object video 108, and the position of the character object video 114 pasted on the decorative frame object video 112 are manually It was necessary to set by. In particular, when the object video 102, the object video 108, and the decorative frame object video 112 are moving images, the position may change from moment to moment and the shape may also change. Therefore, it takes a lot of work to manually paste the logo object video 104, the mask object video 110, and the character object video 114 manually. Therefore, practically, the object video 102, the object video 108, and the decorative frame object video 112 are stored in a storage medium such as a disc, and the object video A102, the object video B108, and the character object video 114 are read while being read by a human. It was necessary to check the position and shape of the object, change the shape of the object video on the material side, and perform the work of pasting it at an appropriate position.
[0005]
[Means for Solving the Problems]
The present invention has been made in order to solve the above-described problems, and includes object video supply means for supplying an object video, shape information extracted from one object video supplied from the object video supply means, and shape information. Output means, material object video storage means for storing a plurality of material object videos to be displayed on the object video supplied from the object video supply means, and object output from the shape information output means Based on the shape information of the video, a detecting means for detecting an area where the material object video is pasted on the object video, and a material object video matching the size of the pasting area detected by the detecting means, the material object Multiple material objects stored in the video storage means A selection means for selecting from among the target video, a pasting means for pasting the material object video selected by the selection means and a target object video to be pasted, and two object videos pasted by the pasting means are displayed. And display means.
[0006]
Further, the selecting means of the present invention selects a material object video close to the size of the pasting area detected by the detecting means from a plurality of material object videos stored in the material object video storing means. The selected material object video is transformed so as to be adapted to the size of the pasting area and output.
[0007]
In the object display device of the present invention described above, the pasting area of the material object is automatically detected based on the shape information of the object video to be pasted. Then, a material object suitable for the detected pasting area is read from the storage means and pasted on the pasting area. Alternatively, the shape of the material object video is deformed, and a material object suitable for the pasting area is pasted on the pasting area. This makes it possible to automate pasting work that has been performed manually and can also handle moving images.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing an embodiment of an object video display device according to the present invention. In the video source 2, a performer of a scene such as a movie or a still life is stored as an object video on a medium such as a disc. In the present invention, the object video stored on the disc will be described for use. However, the object video is extracted in real time from the video captured by the TV camera or the like, and the real-time object video is displayed. May be. The object video read from the video source 2 is output as serially encoded data 4. The serially encoded data 4 is applied to the object video selection unit 6. The object video 100 and the other object video 101 to which the material is attached are classified by the object video selection unit 6. This classification can be realized in advance by adding an index indicating that the object is a target to be pasted at the time of editing. Alternatively, it can be realized by including in the scene configuration information 12 supplied from the video source 2 information on which object video is to be pasted and at what point in time. Alternatively, the user may supply information on the object video to be pasted to the object video selection unit 6 via the setting unit 16.
[0009]
The shape information 8 of the object video 100 is also output from the object video selection unit 6. Here, the shape information is a kind of coordinate information, and is information for knowing the coordinate position where the object exists. Therefore, this shape information includes information on the contour of the object. The shape information is called an alpha plane, and the area where the object video is present is expressed as a logical value 1, and the remaining area which does not exist is expressed as a logical value 0. The coordinates of indicate the remaining area without the object video. In FIG. 1, the object video 100 is assumed to be one sample, but a plurality of samples may be targeted in actual application.
[0010]
The shape information 8 is applied to the pasting area detection unit 18. In the pasting area detection unit 18, pasting information 20 is calculated. The pasting information 20 is information on the pasting area where the material object video on the object video is pasted. This pasting information 20 is applied to the material object deformation unit 26. The material object deforming unit 26 reads a material object video of an appropriate size from the pasting information 20 from the material object storage unit 28. The material object storage unit 28 stores a plurality of material object images such as the names of characters. When there is no material object video of a size that fits the pasting area in the material object storage unit 28, a material object video having a size close to the pasting area is read and enlarged or reduced so as to fit the pasting area. Also do.
[0011]
The material object video 10 generated by the material object deforming unit 26 is applied to the material object pasting unit 30. The original object video 100 is applied to the material object pasting unit 30, and the material object video is pasted on the detected pasting area and output as a processed object video 32.
[0012]
The video composition unit 34 superimposes the object video 101 to which the material object video is not pasted and the processed object video 32 and outputs it as composite serial data 36. The monitor 14 displays a composite video of the object video 101 and the processed object video 32.
[0013]
Next, a method for detecting a region to which a material object video is pasted from a person object video (processing in the pasting region detection unit 18) will be described with reference to FIG. FIG. 2 is a diagram for explaining an example in which a material object (for example, the name of the person) is pasted on the chest of the person object.
[0014]
First, the vertex d of the shape information 8 indicating the outline of the object is detected. In order to detect the head d, the horizontal line 38 is lowered from the upper side with respect to the shape information 8. And the part which contacted the shape information 8 is the top of the head. And it goes along the circumference | surroundings of the shape information 8 toward the lower side from the top of the head. Then, the most depressed portion in the process of tracing is defined as a neck (reference number f).
[0015]
Then, a horizontal line 40 is drawn from the neck. The interval (reference symbol a) between the horizontal line 38 and the horizontal line 40 is measured. a is the vertical size of the head. Next, a horizontal line 42 is drawn downward from the head by αa. αa is a position where the material object image normalized by the size of the head overlaps. That is, when the object video 102 is pulled down depending on the scene, the shape information 8 is also reduced and the head is also reduced. If α is kept constant, αa can be reduced in accordance with the size a of the head.
[0016]
Next, the points g and h where the horizontal line 42 and the shape information 8 intersect are detected. Then, a vertical line is drawn downward from g and h, and a pasting region 24 as shown in FIG. Here, if b is measured, c is determined in the same ratio as b. As described above, even if the coordinates of the shape information 8 move and the size of the shape information 8 itself changes, the pasting area 24 can be detected, and the pasting area 24 is always detected as a similar size. Incidentally, conventionally, it is not easy to detect the position of the lower chest by a predetermined distance from the head of a person. First, the person who appears in the video must be detected. For example, in the MPEG2 method, detection is performed using a motion vector or the like, but if a person is stationary, no motion vector is generated. Therefore, advanced image processing such as identifying the head of the person itself from the background is required. To do. In the present invention, the head can be detected relatively easily from the shape information 8 accompanying the object video by the above-described method, and if the head can be detected, the position of the chest can also be detected easily.
[0017]
Returning to FIG. 1, the pasting region detection unit 18 outputs the detected coordinates of the pasting region 24 and h as pasting information 20, and this is applied to the material object deformation unit 26. The material object deforming unit 26 reads a material having a size suitable from the pasting information 20 from the material object storage unit 28.
[0018]
FIG. 5 is a diagram showing material object videos of various sizes stored in the material object storage unit 28 according to the present invention. FIGS. 5A, 5 </ b> B, and 5 </ b> C show the material object video 10 having the same aspect ratio and size as the pasting area 24 detected by the pasting area detection unit 18. FIG. 5D shows the pasting area 24 currently detected. The material object video 10 having the closest size to FIG. 5D is selected from FIGS. 5A, 5B, and 5C. Since the number of material object videos 10 stored in the material object storage unit 28 is limited, the material object video 10 having the closest size is selected so that the material object deforming unit 26 matches the overlapping area 24. The vertical and horizontal sizes may be modified. When the material is determined by the material object deforming unit 26, the material object video 10 is applied to the material object overlapping unit 30. In the material object duplication unit 30, the material object video 10 is duplicated in a region to be overlapped in the object video 100 and is applied to the video composition unit 34 as a processed object video 32. In the video composition unit 34, the other object video 101 and the processed object video 32, to which the material object video 10 is not pasted, are arranged at predetermined positions on the screen and output as composite serial data 36. Is displayed.
Hereinafter, other embodiments of the present invention will be described.
(Second Embodiment)
A method for detecting a region in which the material object video is pasted on the decorative frame object video (processing in the pasting region detection unit 18) will be described with reference to FIG. Note that the same reference numerals are used for the same reference numerals as those used in FIG.
[0019]
First, a horizontal line 38 in contact with the upper end of the shape information 8 of the decorative frame object image and a horizontal line 40 in contact with the lower end are drawn. Next, a vertical line 44 in contact with the left end of the shape information 8 and a vertical line 46 in contact with the right end are drawn. A rectangle surrounded by the horizontal line 38, the horizontal line 40, the vertical line 44, and the vertical line 46 is detected. Then, the horizontal line 38 is lowered by c and the horizontal line 40 is raised by c. Further, by shifting by d from the right end to the right side and by shifting by d from the left end to the left side, the pasting area 24, that is, a rectangle of b × c is detected. The processing from pasting the object video to compositing and displaying is the same as described above, and will be omitted.
[0020]
Conventionally, the contents of a frame or the like decorated as in this embodiment are manually changed each time. In particular, when the decorative frame moves on the display surface or changes its size, it is difficult to manually insert characters into the decorative frame. According to the present invention, if the appropriate decorative frame object video 112 is selected in the object video selection unit, and the material object video 10 to be pasted is determined by the scene configuration information 12, then the decorative frame object video 112 is automatically displayed. The material object video 10 can be deformed and pasted in a well-balanced manner within the frame. Further, as a feature of the present invention, the material object video 10 can be automatically pasted at a predetermined position even if the size or position of the decorative frame object video 112 changes.
(Third embodiment)
Next, a method for automatically extracting a facial region from a person object video and detecting a region to which a material object video is pasted (processing in the pasting region detector 18) will be described with reference to FIG. Note that the same reference numerals are used for the same reference numerals as those used in FIGS.
[0021]
First, similarly to the method described in FIG. 2, the horizontal line 38 is lowered from the top of the shape information 8 to detect the top d. Then, the outer periphery of the head is traced in the same manner as described above, and the neck portion f is detected. Next, a horizontal line 40 is drawn from f. Then, the vertical line 44 and the vertical line 46 are moved in the outer peripheral direction from the left and right in the section above the horizontal line 40 and below the horizontal line 38, that is, the section a, and stopped at the position in contact with the outer periphery (section b). Thus, a rectangle (axb) is detected. This rectangle roughly surrounds the face. Then, a horizontal line 42 and a horizontal line 48 with a gap of βa in the horizontal direction are drawn, and a vertical line 50 and a vertical line 52 are drawn in the vertical direction from the point where the horizontal line 42 intersects the outer periphery. The rectangle h × i includes important areas such as eyes, nose and mouth of the face portion. A material object video is attached to the rectangular portion. The processing method to be applied is the same as that shown in FIG. The material object video may be a mosaic. Alternatively, the outer peripheral portion may be filtered and blurred with a monochromatic material object video.
[0022]
Conventionally, a face portion is manually selected while watching an image in advance, and mosaic processing is performed on the face. Therefore, when the subject to be processed is moving, the mosaic processing cannot be accurately performed on the face, and the mosaic processing is performed up to the background. However, according to the present embodiment, if an object video to be mosaic-selected is selected, then the face information is automatically detected using the shape information, and the material object video that matches the region is accurately pasted. be able to. The embodiment described above has been described with an example in which an object video is pasted as a pasting target video. However, if the video to be pasted is a video having shape information, the pasted area can be easily detected by the method described above. For the pasted area, a layer structure is used as in the method of creating an animation video, and a bitmap image of the material is placed on the upper layer as the pasting material, and is superimposed on the object video placed on the lower layer. It does not matter if they are combined. As described above, various modifications can be made without departing from the spirit of the present invention.
[0023]
【The invention's effect】
As described above in detail, according to the object video display device of the present invention, the material object paste area is automatically detected based on the shape information of the object video to be pasted, and the material suitable for the detected paste area Since the object is read from the storage means and pasted on the pasting area, the work that has been done manually by the past can be automated and the moving image can be handled.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of an object video display device according to the present invention.
FIG. 2 is a diagram for explaining a method for detecting a region to which a material object video is pasted from an object video of a person.
FIG. 3 is a diagram for explaining a method for detecting an area in which a material object video is pasted on a decorative frame.
FIG. 4 is a diagram for explaining a method of detecting a face area in a person object video.
FIG. 5 is a view showing material object videos of various sizes stored in a material object storage unit;
FIG. 6 is a diagram for explaining an example in which another video is pasted on an object video.
[Explanation of symbols]
2 Video source 6 Object video selection unit 8 Shape information 12 Scene configuration information 14 Monitor 16 Setting unit 18 Pasting area detection unit 20 Pasting information 26 Material object deformation unit 28 Material object storage unit 30 Material object pasting unit 32 Processed object video 34 Video Compositing part

Claims (2)

Object video supply means for supplying object video, means for extracting shape information from one object video supplied from the object video supply means, and outputting it as shape information, and supplied from the object video supply means A material object video storage means for storing a plurality of material object videos to be pasted and displayed on the object video, and a material object on the object video based on the shape information of the object video output from the shape information output means. Detection means for detecting a region to which a video is pasted, and a material object video that matches the size of the pasting region detected by the detection means, among a plurality of material object videos stored in the material object video storage means Selecting means for selecting from the selection An object comprising: pasting means for pasting the material object video selected in the stage and the target object video to be pasted; and display means for displaying the two object videos pasted by the pasting means. Video display device. The selection means selects a material object video close to the size of the pasting area detected by the detection means from a plurality of material object videos stored in the material object video storage means, and selects the selected material The object video display device according to claim 1, wherein the object video is transformed and output so as to match the size of the pasting area.

Images