JP2015050538A - Camera video processing method, apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a camera video processing apparatus that is able to selectively perform video processing only to a background area of an object from a camera video even in a case where the video of the background is not fixed.SOLUTION: An object tracking section 13 tracks an object by performing an image matching process using a template for a camera video. A fitting operation detecting section 15 detects a manual operation for fitting a shape of a skeleton model into a shape of the object. A skeleton model outputting section 16 outputs a skeleton model fitted into the object by being matched with the object. On the basis of the tracking result of the object, a video processing section 17 positions the skeleton model with respect to the camera video. Background processing such as non-gradation processing or mosaic processing is performed for a background area other than an area coinciding with the skeleton model of the camera video. Thereby, the silhouette image of a person rendered invisible other than the object is generated and output.

Description

  The present invention relates to a method, an apparatus, and a program for processing a camera image, and more particularly, to a camera image processing method, an apparatus, and a program for selectively making an image of an object background invisible.

  In video communication such as video conferencing and video chat (conversations conducted while displaying each other's images via the Internet), or on-site coverage when the location of the interview is to be kept secret, only the parties (objects) are output as camera images. There is a desire to make the part invisible by applying a filling process or a mosaic process from the viewpoint of privacy and security.

  In Patent Document 1, as a typical method of object extraction, a background image that does not include an object is acquired in advance on the assumption that the camera is fixed, and the difference between the captured video and the background image is obtained as an object region. Is extracted, and the image processing is performed using the other region as the background portion.

Japanese Patent Application No. 2012-214194

  Since the method proposed in Patent Document 1 is based on the premise that the background image is fixed, that is, the camera is fixed, it is necessary to acquire the background image again when the camera is moved, and smooth communication is performed. There was a problem of inhibiting. In particular, it could not be applied when the camera always moves like a handy type.

  On the other hand, the face detection function installed in smart device cameras, etc. can identify rough face areas, but cannot recognize the outline of the face or body (object), thus causing image loss in the object. In addition, it was not possible to accurately recognize only the background area and perform image processing.

  An object of the present invention is to solve the above technical problem and provide a camera video processing method, apparatus, and program capable of performing video processing only from the camera video to the object background area even when the background video is not fixed. It is in.

  In order to achieve the above object, a camera image processing apparatus according to the present invention is characterized by having the following configuration.

  (1) Object tracking means for tracking an object in the camera image, skeleton model output means for outputting a skeleton model imitating the outline of the object, and positioning the skeleton model at the object tracking position and outputting an object silhouette image And an image processing means for detecting the fitting operation on the skeleton model and reflecting the result on the shape of the skeleton model.

  (2) The skeleton model is provided with an α blend area in the outline portion, and the image processing means semi-visualizes the camera image in an area overlapping the α blend area.

According to the present invention, the following effects are achieved.
(1) Since the shape of the skeleton model can be fitted to the contour of the object, even if there is an individual difference in the shape of the object, it is possible to provide an object silhouette video in which only the background of the object is accurately invisible.

  (2) The display position of the skeleton model moves according to the tracking result of the object, and the relative positional relationship between the object and the skeleton model is always maintained, so even if the object or background moves in the camera image, It is possible to provide an object silhouette image in which only the background of the object is accurately invisible.

  (3) Since the vicinity of the contour of the skeleton model is an α blend region and the contour portion of the object can be an α blend image, the background can be reliably invisible without impairing the shape of the object.

It is a functional block diagram of the camera video processing apparatus which concerns on one Embodiment of this invention. It is the flowchart which showed operation | movement of one Embodiment of this invention. It is the figure which showed an example of the camera image | video. It is the figure which showed an example of the face recognition result which made the camera image | video object. It is the figure which showed an example of the template. It is the figure which showed an example of the skeleton model. It is a figure showing an example of a person silhouette picture (before fitting). It is a figure showing an example of a person silhouette picture (after fitting). It is the figure which showed an example of the person silhouette image | video at the time of object tracking.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of the main part of a camera video processing apparatus according to an embodiment of the present invention, and here, the configuration unnecessary for the description of the present invention is omitted.

  The camera video processing apparatus according to the present embodiment may be configured by mounting applications (programs) for realizing each function on a general-purpose computer (server, smartphone, PDA), or a part of the applications may be implemented as hardware. Alternatively, it may be configured as a ROM-only dedicated machine or a single-function machine.

  The object detection unit 11 detects an object from the camera video output by the imaging unit 10 in units of frames. In the present embodiment, description will be made assuming that the object detection unit 11 performs face detection and a human face region is detected as an object.

  The template registration unit 12 registers a rectangular area including a face area (object) as a template. The object tracking unit 13 tracks an object by performing image matching processing (template matching) using the template for a camera video.

  In the skeleton model storage unit 14, a basic shape of a skeleton model imitating the skeleton shape of an object is registered in advance. FIG. 6 is a diagram showing an example of the basic shape of the skeleton model. If the object to be tracked is a human, for example, the skeleton region R1 (black portion) corresponding to the contour shape of the upper body and the face of the skeleton region An α blend region R2 (gray portion) provided with a predetermined width along the contour of the region.

  The fitting operation detection unit 15 detects a manual operation for fitting the shape of the skeleton model to the shape of the object. The skeleton model output unit 16 reads out and outputs the basic shape of the skeleton model from the skeleton model storage unit 14 until the fitting operation is detected, and when the fitting operation is detected, the fitting operation is converted into the skeleton model. The skeleton model after fitting obtained by reflecting is output.

  The video processing unit 17 includes processing units 17a and 17b that are made invisible by painting an object background or performing mosaic processing, and the skeleton model output unit 16 outputs the camera video output by the imaging unit 10. Based on the tracking result output by the object tracking unit 13, the skeleton model is positioned so that the center coordinates of the object coincide with the center coordinates of the skeleton model. Then, by performing background processing such as filling processing and mosaic processing on the background region other than the region that overlaps the skeleton model of the camera image, a person silhouette image (FIGS. 8 and 9 to be described later) in which objects other than the object are made invisible is generated. ,Output.

  In this embodiment, since the skeleton model is configured by the skeleton region R1 and the α blend region R2, the camera image of the region overlapping the skeleton region R1 is visualized, and the camera image of the region overlapping the α blend region R2 is semi-visualized, Other areas are invisible.

  Next, the operation of the embodiment of the present invention will be described with reference to the flowchart of FIG. In step S1, as shown in FIG. 3, camera images output from the imaging unit 10 are sequentially acquired in units of frames. In step S2, the object detection unit 11 performs face recognition for the time series of the camera video, and detects an object (a face area in this embodiment) as shown in FIG.

  In step S3, as shown in FIG. 5, a rectangular area including a face area is registered in the template registration unit 12 as an object template. In step S4, the object tracking unit 13 performs template matching for the camera image using the template, and starts object tracking in which a region having a high matching likelihood is determined as an object and sequentially tracked.

  In step S <b> 5, the skeleton model registered in the skeleton model output unit 16 is taken into the video processing unit 17. In this embodiment, since the basic shape of the skeleton model is registered in advance in the skeleton model output unit 16, the basic shape skeleton model is captured here.

  In addition to adding a person identification function to the face recognition function, a skeleton model unique to each identification candidate is prepared in advance so that a corresponding skeleton model is selectively captured based on the person identification result. Also good. In this way, if the skeleton model can be selected for each person, the contours such as the body shape and hairstyle, the person who wants to make the exposed part only the face and the person who can tolerate not only the face but also the upper body are mixed. Even if it is, the basic shape of the skeleton model can be optimized, so that the fitting operation described later becomes easy.

  In step S6, the video processing unit 17 causes the skeleton model (pre-fitting) to be performed on the camera video so that the center coordinate of the face area being tracked in the camera video matches the center coordinate of the face area of the skeleton model. ) Is positioned and superimposed and output as a person silhouette image.

  FIG. 7 is a diagram showing an example of a person silhouette image. At this point, the skeleton model remains the basic model before fitting to the object, so there is a gap between the object outline and the skeleton model outline. In the vicinity of the outline, the background is not sufficiently invisible.

  In step S <b> 7, the fitting operation detection unit 15 determines whether or not the operator has performed a fitting operation on the skeleton model. Here, when the operator performs an operation of fitting the skeleton model to the contour shape of the object as shown in FIG. 8 by appropriately operating an input operator such as a mouse, a touch panel, or a keyboard, this is detected. Then, it progresses to step S8.

  In step S8, the skeleton model after fitting is updated and registered in the skeleton model output unit 16. Accordingly, since the skeleton model after the fitting is superimposed on the camera image that is being tracked thereafter, a person silhouette image in which only the background region is accurately invisible or semi-visualized is output. .

  In step S9, based on the likelihood of template matching in step S4, it is determined whether or not the object can be normally tracked. If the tracking is normally performed, the process returns to step S4, and the object tracking, the positioning of the skeleton model, and the output of the person silhouette video are repeated. On the other hand, when the matching likelihood falls below a predetermined threshold value, it is determined that tracking has failed, and the process returns to step S2, and each of the above processes is performed again from the detection of the object.

  FIGS. 9A, 9B, and 9C are diagrams showing output examples of the camera video and the person silhouette video when the object tracking is normally performed, while suppressing the lack of display of the object. You can see how the background is accurately invisible or semi-visible.

  In the above-described embodiment, the shape of the skeleton model (FIG. 6) prepared in advance has been described as imitating the upper body and head of a person, but the present invention is not limited to this. Instead, for example, a large number of skeleton models including a skeleton model imitating the whole body of a person may be prepared so that the operator can select a desired skeleton model according to the shape of the object in the camera video.

  In the above embodiment, the object is described as being a human, but the present invention is not limited to this, and may be a tangible part (article) other than a human. In this case, an algorithm for recognizing the article may be employed instead of the recognition algorithm for the face recognition, and a model that imitates the contour shape of the article may be prepared as the skeleton model.

  According to the present embodiment, since the shape of the skeleton model can be fitted to the contour of the object, even if there is an individual difference in the shape of the object, it is possible to provide an object silhouette image in which only the background of the object is accurately invisible. It becomes like this.

  In addition, the display position of the skeleton model moves according to the tracking result of the object, and the relative positional relationship between the object and the skeleton model is always maintained, so even if the object or background moves in the camera image, the object It is possible to provide an object silhouette image in which only the background of the object is accurately invisible.

  Furthermore, since the vicinity of the outline of the skeleton model is an α blend area and the outline portion of the object can be an α blend image, the background can be reliably invisible without impairing the shape of the object.

  DESCRIPTION OF SYMBOLS 10 ... Imaging part, 11 ... Object detection part, 12 ... Template registration part, 13 ... Object tracking part, 14 ... Skeleton model memory | storage part, 15 ... Fitting operation detection part, 16 ... Skeleton model output part, 17 ... Video processing part

Claims (8)

An object tracking means for tracking an object in a camera image;
A skeleton model output means for outputting a skeleton model imitating the contour of an object;
Image processing means for positioning the skeleton model at the tracking position of the object and outputting an object silhouette image;
A camera image processing apparatus, comprising: a fitting operation detection unit that detects a fitting operation on the skeleton model and reflects the fitting operation on the shape of the skeleton model. A template registration unit for registering a template of an object detected from the camera image;
The camera image processing apparatus according to claim 1, wherein the object tracking unit tracks an object by template matching with respect to the camera image.   3. The camera image processing apparatus according to claim 1, wherein the image processing unit visualizes a camera image in an area overlapping with the skeleton model and invisible other areas. 4. The skeleton model has an α blend region in its contour portion,
The camera image processing apparatus according to claim 3, wherein the image processing unit semi-visualizes a camera image in a region overlapping with the α blend region. Means for recognizing a face area from a camera image;
Means for identifying a person whose face has been recognized,
The skeleton model is prepared for each person as an identification candidate,
The camera image processing apparatus according to claim 1, wherein the skeleton model output unit outputs a skeleton model corresponding to the identified person.   6. The camera video processing apparatus according to claim 1, wherein the camera video processing apparatus is configured by a computer or a server. In a camera image processing method executed by a computer,
Procedures for tracking objects in the camera image,
A procedure for positioning the skeleton model imitating the outline of the object at the tracking position of the object and outputting the object silhouette video,
And a procedure for detecting a fitting operation on the skeleton model and reflecting the detected operation on the shape of the skeleton model. Procedures for tracking objects in the camera image,
A procedure for positioning the skeleton model imitating the outline of the object at the tracking position of the object and outputting the object silhouette video,
A camera image processing program for causing a computer to execute a procedure of detecting a fitting operation on the skeleton model and reflecting the detected operation on the shape of the skeleton model.