KR101652022B1 - Method, apparatus and computer-readable recording medium for segmenting a second obejct which is overlapped with a first object in image - Google Patents

Abstract

According to an aspect of the present invention, there is provided a method for separating a second object superimposed on a first object in an image, the method comprising the steps of: (a) determining whether the second object is an image A first edge included in the superimposed image among the extracted edges of the superimposed image and the template for the image of the first object in a state in which the second object is not overlapped with the first object is identified ; (b) the image processing apparatus judges whether or not at least one edge pixel of the second edge exists in a predetermined area including edge pixels of the first edge; And (c) the image processing apparatus determines an edge pixel of the first edge as one of a pixel of the first object and a pixel of the second object based on whether at least one edge pixel of the second edge exists The method comprising the steps of: According to the present invention, it is possible to easily and stably separate the first object and the second object having similar colors.

Description

METHOD, APPARATUS AND COMPUTER READABLE RECORDING MEDIUM FOR SEGMENTING A SECOND OBJECT WHICH IS OVERLAPPED WITH A FIRST OBJECT IN IMAGE FOR DETACHING A SECOND OBJECT NAMED INTO A FIRST OBJECT IN AN IMAGE,

The present invention relates to a method, an apparatus and a computer-readable recording medium for separating a second object superimposed on a first object in an image. More particularly, the present invention relates to a method, apparatus and computer- Determining whether a second edge of the face template is present in a predetermined area including an edge pixel of the first edge and determining whether a second edge of the face template exists in an edge of the edge of the first edge, An apparatus and a computer-readable recording medium that enable a user to separate a hand overlapping a face in an image by determining the pixel as a pixel of the area of the face or a pixel of the area of the hand.

A variety of applications for recognizing the user's hand movements, or gestures, have been presented using cameras. The application accurately recognizes the meaning of the gesture of the user, and executes a command or process corresponding to the recognized meaning. The recognition of these gestures has been mainly studied in the field of human-computer interaction. Various attempts have been made to recognize gestures such as remote control using a smartphone, communication, and execution of games and applications.

However, although various methods of recognizing a user's gesture using a camera have been proposed, recognition of such a gesture may be limited or inaccurate in some cases. For example, if the user's hand overlaps with the user's face or another object having a color similar to the user's skin color, it is difficult to separate the user's hand from the face or other object in the information of the image acquired by the camera, Therefore, there is a problem that the user can not accurately recognize the actual gesture.

Korean Patent No. 10-0826878 and the like have been disclosed in relation to recognition of a user's body such as a face and a hand. However, these applications and patents do not provide a direct solution to the problem associated with recognizing hand superimposed on the user's face as described above.

Thus, the present inventor has found that by determining an edge pixel as a pixel of a face area or a pixel of a hand area with respect to an edge pixel of an area of a face where a user's hand is superimposed on a face, A technique for separating hands has been developed and a technology for separating a second object superimposed on a first object has been developed by generalizing a technique of separating a hand superimposed on a face as described later .

It is an object of the present invention to solve all the problems described above.

In addition, in recognizing the user's gesture, the present invention stably separates the area of the face and the area of the hand with only a small amount of calculation when the user's hand overlaps the user's face in the photographed image, So that the meaning of the operation and the operation can be accurately recognized.

It is another object of the present invention to allow a hand to be separated from an image in which a face and a hand are superimposed using an edge gradient orientation of an edge pixel in an image superposed with a face template.

Another object of the present invention is to stably separate a second object superimposed on a first object even if the color information of the first object and the second object are similar.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an embodiment of the present invention, there is provided a method for separating a second object superimposed on a first object in an image, the method comprising the steps of: (a) And a first edge included in the superimposed image among the edges extracted in a state in which the template for the image of the first object in a state in which the second object is not superimposed on the first object is combined Identifying; (b) the image processing apparatus judges whether or not at least one edge pixel of the second edge exists in a predetermined area including edge pixels of the first edge; And (c) the image processing apparatus determines an edge pixel of the first edge as one of a pixel of the first object and a pixel of the second object based on whether at least one edge pixel of the second edge exists The method comprising the steps of:

According to an aspect of the present invention, there is provided an image processing apparatus for separating a second object superimposed on a first object in an image, the apparatus comprising: a superimposed image in which a second object is an image superimposed on a first object; Identifies a first edge included in the superimposed image among edges extracted in a state in which a template for an image of the first object is not overlapped with the first object, An image processing unit for determining whether at least one edge pixel of the second edge exists in a predetermined area including the pixel; And an area determining unit that determines an edge pixel of the first edge as one of the pixel of the first object and the pixel of the second object based on whether at least one edge pixel of the second edge exists do.

In addition, there is further provided another method, apparatus, system for implementing the invention and a computer readable recording medium for recording a computer program for executing the method.

According to the present invention, in recognition of a gesture of a user required in a game machine and other devices, it is possible to stably separate hands superimposed on a user's face and a face in a captured image, have.

In addition, according to the present invention, when the user's hand is superimposed on the user's face in the image of the movement of the user, the area of the face and the area of the hand can be stably separated and can be performed with only a small amount of calculation.

According to the present invention, even if the color information of the first object and the second object is similar, the second object superimposed on the first object can be stably separated and can be performed with only a small amount of computation.

1 is a view schematically showing the configuration of an image processing apparatus 100 for separating and identifying a second object superimposed on a first object in an image according to the present invention.
2 is a flowchart of a method for separating and identifying a hand from a face in an image in which a user's hand is superimposed on a user's face in accordance with the present invention.
3 is a view for explaining a skin color model according to an embodiment of the present invention.
FIG. 4 illustrates a method for determining whether a user's hands and faces are overlapped according to an embodiment of the present invention.
FIG. 5 shows the number of pixels of a skin color varying depending on whether a hand or a face is overlapped according to an embodiment of the present invention.
FIG. 6 illustrates a method of determining whether another edge exists in a predetermined area including an edge pixel of a superimposed image according to an embodiment of the present invention.
Figure 7 illustrates a method of classifying an identified edge pixel according to an embodiment of the present invention into one of the pixels of the face area and one of the pixels of the hand area.
8 (a) to 8 (j) show drawings for explaining a method of classifying an edge pixel according to an embodiment of the present invention into one of a pixel of a face area and a pixel of a hand area.
FIG. 9 illustrates an example of a result of separating a hand edge from a superimposed image according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

Configuration of image processing device

1 is a view schematically showing the configuration of an image processing apparatus 100 for separating and identifying a second object superimposed on a first object in an image according to the present invention. It is assumed that the first object and the second object have similar color information. In an embodiment of the present invention, assume that the first object is a face and the second object is a hand in order to facilitate understanding. I want to explain. It is also clear that the present invention extends not only to separating the hands overlapping the face but also to generalizing and separating the second object superimposed on the first object. The similarity of the color information of the first object and the second object may mean that the Mahalanobis distance on the color modeling is smaller than the predetermined threshold value. However, the present invention is not limited to this, 2 It may be assumed that the object's color, brightness, saturation, etc. are close to the predetermined conditions.

The image processing apparatus 100 may include a color processing unit 110, an image processing unit 120, and an area determination unit 130.

According to an embodiment of the present invention, the color processing unit 110, the image processing unit 120, and the area determining unit 130 may be program modules in which at least a part of them communicate with an external system. Such program modules may be included in the image processing apparatus 100 in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known memory devices. In addition, at least some of these program modules may be stored in a remote storage device capable of communicating with the image processing apparatus 100. [ Such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or perform particular abstract data types as described below in accordance with the present invention.

First, the color processing unit 110 can define a skin color model. The image processing apparatus 100 can identify the area of the face in the image based on the defined skin color model. For reference, in the case of separating a second object superimposed on the first object, there will be color modeling for colors corresponding to the first object and the second object instead of the skin color model.

The image processing unit 120 can identify the edge in the image in which the user's hand and face are superimposed. In addition, the image processing unit 120 can determine whether another edge of the face template exists in a predetermined area including the edge pixels of the identified edge. Here, the face template may be a face image of the user in a state in which the face of the user and the user's hand are not overlapped (for reference, when it is generalized to separate a second object superimposed on the first object, You can assume a template). In addition, the image processing unit 120 may generate the edge gradient orientation of the edge pixels.

The area determination unit 130 may determine the edge pixels of the identified edge as pixels of the face area or pixels of the hand area based on whether or not another edge of the face template exists in the predetermined area. Specifically, the area determining unit 130 may determine the edge pixel based on the edge-gradient orientation of the edge pixel as one of the pixels of the area of the face excluding the overlapped hand and the area of the overlapping hand. The area determination unit 130 can separate the area of the hand superimposed on the area of the face by determining the edge pixel as the pixel of the area of the face or the pixel of the area of the hand.

Specific functions of the color processing unit 110, the image processing unit 120, and the area determination unit 130 will be described in detail below with reference to FIG. 2 to FIG.

A method for determining an area of a hand.

2 is a flowchart of a method for separating and identifying a hand from a face in an image in which a user's hand is superimposed on a user's face in accordance with the present invention.

(i) The color processing unit 110 can define a skin color model (S210).

According to an embodiment of the present invention, the color processing unit 110 may transform an RGB color profile of an image into a YCbCr form. The image processing unit 120 can set the unique color gamut of the object of the image using the transformed YCbCr shape and set the edge of the object and the edge gradient orientation of the edge pixel. In addition, the color processing unit 110 can determine or decompose a skin color region using CbCr. For example, an object in an image may have a unique CbCr distribution. The image processing apparatus 100 can determine whether the object corresponds to the user's body according to the CbCr distribution of the object and can determine the area of the user's face.

In an embodiment of the present invention, the color processing unit 110 may use a median filter in the above modification. The color processing unit 110 can protect the edge by using an intermediate filter and can remove outliers in the skin color region. The intermediate filter may have a predetermined window size.

According to an embodiment of the present invention, the color processing unit 110 may generate a skin color model using simple training images during the initialization process.

The image processing unit 120 can identify an area representing a user's body, that is, a face area, of the image based on the skin color model. With respect to the identification of the area of the face, the skin color model according to one embodiment of the present invention will be described in detail below with reference to Fig.

 (ii) The image processing unit 120 can acquire a face template (S220).

In addition, the image processing unit 120 may store a face template.

The image processing apparatus 100 may process a series of images that are temporally sequential. Depending on the movement of the user, the series of images can be classified into an image in which the user's face and hands are not overlapped and an image in which the user's face and hands are superimposed. That is, the face template may be an image including an area of a face before the user's hand is superimposed on the face. Further, the face template may be an object that is compared with an image in which the user's hand is superimposed on the face. Hereinafter, an image in a state in which a user's hand is superimposed on a user's face is outlined as a superimposed image.

The face template may include one or more face regions, and the image processing unit 120 may identify one or more face objects based on the skin color model. According to an embodiment of the present invention, the position of the face may be determined by skin segmentation based on the skin color model.

Also, according to an embodiment of the present invention, the texture of the area of the face may be stored as a face template, and when the information of the edge is generated by performing the step S225 described later, the edge orientation is stored as a face template It is possible.

(iii) The image processing unit 120 may generate the information of the edge of the face template using the stored face template (S225).

In one embodiment of the invention, the face template may comprise an edge. The image processing unit 120 may extract an edge from a face template and a superimposed image to be described later, for example, using a Canny edge detector.

In one embodiment of the present invention, as described above, the face template may be a face image of the user without the user's hands overlapping. The image processing unit 120 may generate and store information of an edge, information of an edge gradient of an edge pixel of an edge, and information of a gradient orientation of an edge pixel for each edge of one or more edges of the face template .

(iv) The image processing unit 120 may determine whether the user's hand is superimposed on the face in the image (S230). If the user's hand is not superimposed on the face, the image processing unit 120 may determine that the image to be judged is a face template, and the above-described step S220 may be repeated. If the user's hand is superimposed on the face, the image processing unit 120 may determine that the image subjected to the judgment is an overlaid image, and the following step S240 may be performed next.

In the following, a specific method for determining whether or not a user's hand is superimposed on a face in an image according to an embodiment of the present invention will be described in detail with reference to FIG.

(v) The image processing unit 120 may identify the first edge of the superimposed image with the hand superimposed on the face (S240). The first edge may represent one or more edges of the face and the area of the hand with the hand superimposed on the face.

(vi) The image processing unit 120 may generate information of an edge gradient of one or more edge pixels of the first edge using the information of the identified first edge (S245).

The image processing unit 120 can calculate the edge gradients of the edge pixels and store the information of the calculated edge gradients for each edge of one or more edges of the superimposed image.

(vii) The image processing unit 120 may determine whether there is at least one other edge in a predetermined area including the edge pixels of the first edge of the superimposed image (S250). At this time, it can be assumed that the existing face template and the overlapped image are overlapped, or whether or not another edge exists in a state comparable to the existing face template. That is, one can imagine an area of the combined image comprising one or more edges of the face template and one or more edges of the superimposed image.

In one embodiment of the present invention, the predetermined region may be a region having a predetermined size centered on the position of the edge pixel of the first edge. For example, an area having a predetermined size may be a rectangular area of 3x3 or 5x5 pixels. That is to say, the image processing unit 120 can determine whether or not there is another edge within a predetermined area of the face template centered on the position of the edge pixel of the first edge.

If there is no other edge of the face template in the predetermined area including the edge pixel of the first edge (i.e., centered on the edge pixel of the first edge), step S260 may be performed, If another edge exists in a predetermined area including edge pixels of the edge, step S255 may be performed.

In one embodiment of the present invention, depending on whether or not another edge exists in a predetermined area including edge pixels of the first edge of the superimposed image, the area determining unit 130 sets the edge pixels of the first edge to the face Or an area of a hand, or when at least one other edge exists in a predetermined area including an edge pixel of the first edge of the overlapped image, the edge determination unit 130 may classify the edge pixels of the first edge into pixels of a face area or a hand area (S260).

As described above, the face template may be the face image of the user's face and the hand without overlapping. Thus, if there is a second edge of the face template within a predetermined area determined based on the position of the edge pixel of the first edge identified in the area of the face of the superimposed image, then the edge pixel of the identified first edge is the face May be considered to be relatively high. On the other hand, if there is no second edge of the face template in the predetermined area determined based on the position of the edge pixel of the identified first edge, the edge pixels of the first edge are overlapped (not shown in the face template) The likelihood of being a pixel of a hand may be considered relatively high.

A method of determining whether or not a second edge of a face template exists in a predetermined area including an edge pixel of the first edge will be described in detail with reference to FIGS. 6 and 7 below.

On the other hand, when at least one other edge exists in a predetermined area including an edge pixel of the first edge of the superimposed image, the area determining unit 130 determines that the edge of the first edge of the first edge Is classified into a face region or a hand region region, the following description will be made.

In one embodiment of the present invention, the area determination unit 130 determines the edge pixel of the first edge based on the difference value between the edge gradient orientation of the edge pixels of the first edge and the edge gradient orientation of the other edge pixels, Pixel or a pixel of a region of the hand. For example, when the difference value between the edge gradient orientation of the first edge pixel and the edge gradient orientation of at least one other pixel corresponding to the edge pixel of the first edge is less than or equal to a predetermined value, The edge pixel can be classified into the pixels of the area of the face and the first edge pixel can be determined as the pixel of the area of the hand if the difference value is larger than the predetermined value (refer to FIG. 7 Described later). Here, at least one other pixel corresponding to the first edge pixel of the superimposed image may be another edge pixel adjacent to the edge pixel of the first edge within a predetermined distance. Also, at least one other pixel corresponding to the first edge pixel may comprise an edge pixel of the second edge on the face template, in which case it may be an edge pixel in a predetermined area of the face template closest to the first edge pixel have. The predetermined area may be an area of the face template determined based on the position of the first edge pixel described above in step S250.

(viii) If the edge pixels are classified into one of the pixels of the face region or the pixels of the hand region, the region determining unit 130 determines whether the change between the superimposed image and the face template satisfies a predetermined criterion May be added to the area of the overlapping hands (S270).

When overlapping by hand occurs, a change in color, brightness, etc. may occur in the area of the face of the superimposed image as compared with the area of the face of the face template. For example, due to the difference between the color of the face and the color of the hand, a change in color may occur in a part of the area of the face of the superimposed image. Alternatively, a change in brightness may occur in a portion of the face region of the superimposed image. The area determining unit 130 can refer to the above change and can more accurately determine the boundary of the area of the overlapping hands.

In one embodiment of the present invention, the area determining unit 130 determines whether or not the area of the face in which the hand is superimposed on the face is overlapped with the face template and the predetermined part in which the difference in color between the superimposed images is equal to or larger than the predetermined value Can be added to the area of the hand. Here, the predetermined portion may be limited to a portion adjacent to or adjacent to the edge pixel determined as the area of the overlapping hand. Further, the predetermined portion may be limited to a portion or pixel whose distance from the edge pixel determined as the region of the overlapping hand is equal to or less than a predetermined value. The area determining unit 130 can add a predetermined portion of the area of the face where the hand is superimposed on the face with the difference in brightness between the face template and the superimposed image to a region of the overlapping hands have.

(ix) The area determining unit 130 may supplement the information of the lost edge in the superimposed image (S280).

The area of the hand may be an area surrounded by the edge pixels determined by the edge pixels of the area of the overlapping hands and may be an area including the area added in step S270.

Some of the edge pixels of the area of the overlapping hands may be lost in the overlapping image and the overlapped hands in step S260 may be misclassified as pixels of the area of the face that is excluded. The area determining unit 130 can finally determine the contour (i.e., outline) of the area of the overlapping hands. For example, the region determining unit 130 may further classify edge pixels or regions between edge pixels determined as pixels of the region of the overlapping hands as pixels of the overlapping region of the hand, The outline of the region to be overlapped can be further classified into the edges of the overlapping hands.

A skin color model

3 is a view for explaining a skin color model according to an embodiment of the present invention.

As shown in FIG. 3, the skin color distribution in the characteristic space can be modeled as a Bivariate normal distribution. In Figure 3, the standard deviation σ can be expressed. The point 310 may represent the center value of the skin color, and the region 320 may represent a region corresponding to the skin color.

C may be a feature vector by C _b and C _r of a pixel and may be a 2-D (dimensional) Euclidean space R ² Or may be a vector in the feature space S. C can be defined as: " (1) "

mu _s denotes the center value, and when S denotes the covariance, the Mahalanobis distance d from C to _s can be defined as Equation 2 below.

That is to say, as described in equation (2), the skin color model can provide a mapping from the 2D feature space to the 1D distance space. The smaller the value of d, the more likely it is that c is closer to skin color.

According to equation (2), when d is a constant, the outline of the constant density (that is, area 320) may be an ellipse. Principal axes of these ellipses can be given by eigenvectors of Σ s . In addition, the eigenvalues of S can determine the lengths of the axes.

The image processing unit 120 may determine whether the object is indicative of the skin based on the measured distance between the feature vector of the object and μ _s . The object may include regions, edges, pixels, etc. of the image.

In one embodiment of the present invention, the distance may be the Mahalanobis distance described above, but is not limited thereto.

If the measured distance is smaller than a predetermined threshold value, the image processing unit 120 may determine that the object represents skin of the face. When the measured distance is equal to or greater than a predetermined threshold value, the image processing unit 120 may determine that the object does not represent the skin of the face.

Determining whether the user's hand or face is overlapping

FIG. 4 illustrates a method for determining whether a user's hands and faces are overlapped according to an embodiment of the present invention.

The image processing unit 120 can detect the area 410 of the face in the image 400 and set the first inspection area 420 and the second inspection area 430 as the aforementioned boundary.

In one embodiment of the present invention, the first examination area 420 may be a polygon or a circle of the minimum size including the area 410 of the face. The second inspection region 430 may be a region larger than the first inspection region 420 including the first inspection region 420. For example, the second examination area 430 may be an area that may further include the user's ear or neck as compared to the first examination area 420. [ Alternatively, the second inspection region 430 may be a region having a predetermined size or a predetermined number of pixels in each of the upper, lower, left, and right regions, as compared with the first inspection region 420.

Some of the pixels of the first inspection region 420 may correspond to the skin color. In addition, some of the pixels of the second inspection region 430 may correspond to the skin color. Here, the pixel corresponding to the skin color may be a pixel having a probability that the skin color is a predetermined value or more, and the pixel having the distance d described above with reference to Fig. 3 may be a predetermined value or less.

According to an embodiment of the present invention, when the hand and the face are not overlapped, it can be seen that there is no user's body such as an arm or the like between the first examination region 420 and the second examination region 430 . It is assumed that the number of pixels corresponding to the skin color of the first inspection region 420 and the number of pixels corresponding to the skin color of the second inspection region 430 will not greatly differ when the hands and the face are not overlapped . On the other hand, when the hand and the face are overlapped, the user's body such as the hand and the arm may be present between the first inspection region 420 and the second inspection region 430. Thus, the image processing unit 120 may determine that the area of the face 410 in the image 400 is located at a position where the hand is superimposed on the face (e.g., State or not. Specifically, the image processing unit 120 calculates the difference between the number of pixels corresponding to the skin color of the first inspection region 420 and the number of pixels corresponding to the skin color of the second inspection region 430, It is possible to judge whether or not the hand and the face are overlapped. The image processing unit 120 may determine that the face region 410 is in a state where the hand is superimposed on the face when the difference is greater than or equal to a predetermined threshold value and if the difference is smaller than the predetermined threshold value, It can be determined that the area 410 is in a state where the hand is not superimposed on the face. In one embodiment of the present invention, the difference between the size of the first inspection region 420 and the size of the second inspection region 430 and the above-mentioned threshold value indicates whether or not the hand is in a state in which no face is overlapped, As shown in Fig.

As described above in the description of FIG. 2, the image processing apparatus 100 may process a series of temporally sequential images. A series of images may be classified into an image in which the hand is not superimposed on the face (e.g., an image to be used as a face template) and an image in which a hand is superimposed on a face (e.g., an image to be used as a superimposed image) . In Fig. 5 below, a series of images in which whether or not the hand and face are superimposed change with time, are illustrated.

FIG. 5 shows the number of pixels of a skin color varying depending on whether a hand or a face is overlapped according to an embodiment of the present invention.

5, the x-axis represents time and the y-axis represents the number of pixels corresponding to the skin color.

As shown in FIG. 5, the first state on the left is a state in which no hand is superimposed on the face region, and a hand exists outside the second examination region 430. In this case, the first examination region 420 The curve 520 indicating the number of pixels corresponding to the skin color included in the second inspection area 430 and the curve 510 indicating the number of pixels corresponding to the skin color included in the second inspection area 430 have.

Next, the second state from the left is a state in which the hand enters the second inspection area 430 immediately before the hand is superimposed on the face area. In this case, the skin color of the first inspection area 420 A curve 510 indicating the number of pixels corresponding to the skin color included in the second inspection area 430 and a curve 510 indicating the number of pixels corresponding to the skin color included in the second inspection area 430 It can be seen that the number of pixels represented by the curve 520 is maintained to be similar to that of the curve 520, which is larger than the curve 520.

Next, the third state from the left is a state in which the hand is superimposed on the face region and the hand enters the first inspection region 420. In this case, the skin color included in the first inspection region 420 When comparing the curve 520 representing the number of corresponding pixels and the curve 510 representing the number of pixels corresponding to the skin color included in the second inspection region 430, And the number of pixels represented by the pixel 520 is significantly larger than the number of pixels.

Next, the fourth state from the left is a state immediately after the state in which the hand is superimposed on the face area is released, and there is almost no hand inside the first inspection area 420, and the inside of the second inspection area 430 In this case, a curve 520 indicating the number of pixels corresponding to the skin color included in the first examination region 420 and a curve 520 indicating the number of pixels corresponding to the skin color included in the second examination region 430 It can be seen that the number of pixels represented by the curve 510 is larger than the number of pixels represented by the curve 520 and is substantially similar to the curve 520. In this case,

Next, the fifth state from the left is a state in which the hand is not superimposed on the face region and the hand completely exits to the outside of the second examination region 430. In this case, the fifth examination region 420 included in the first examination region 420 The difference between the curve 520 indicating the number of pixels corresponding to the skin color and the curve 510 indicating the number of pixels corresponding to the skin color contained in the second examination region 430 is almost not different.

That is, the third and fourth states in which the number of pixels represented by the curve 510 is larger than the number of pixels represented by the curve 520 by a predetermined threshold or more are judged to be images in which the hand and the face are overlapped.

That is, in one embodiment of the present invention, the image processing unit 120 calculates the difference between the number of pixels corresponding to the skin color of the first inspection region 420 and the number of pixels corresponding to the skin color of the second inspection region 430 Based on which the start and end of overlapping of hands and faces among a series of images can be detected.

Edge's  Classification

FIG. 6 illustrates a method of determining whether another edge exists in a predetermined area including an edge pixel of a superimposed image according to an embodiment of the present invention.

6, the edges 611, 612, 613 and 614 of the superimposed image are indicated by solid lines, and the edges 621 and 622 of the face template are indicated by dotted lines.

In Fig. 6, the edges 612 and 613 are edges corresponding to the hand (or finger). That is to say, the edges 612 and 613 are the edges including the edge pixels to be determined as the outline of the hand superimposed in the step S260 described above with reference to Fig. In addition, edges 611 and 614 are edges that include edge pixels that are to be determined as the outline of the face. A face template is a face image in which the hand is not superimposed on the face. Thus, for edges pixels of edges 611 and 614, there are edges 621 and 622 of the corresponding face template, while for edges pixels of edges 612 and 613, the edges of the corresponding face template are Most do not exist.

As described above with reference to FIG. 2, the image processing unit 120 can determine whether another edge exists in a predetermined area determined by referring to a specific edge pixel of the edge of the superimposed image. In FIG. 6, adjacent regions are illustrated as a 5x5 rectangle (or window) by way of example, but it is experimentally found that the best performance is achieved with a 3x3 rectangle.

As shown in FIG. 6, there is another edge in an area 631 (i.e., a window) that includes edge pixels 630. In other words, in the area 631, there is an edge 621 of the face template. On the other hand, in the region 641 including the edge pixel 640, there is no other edge. Thus, the edge pixel 630 may be classified as an edge pixel having another edge in a predetermined area including the edge pixel 630 in the above described step S250 with reference to FIG. 2, and in step S255 An edge-gradient orientation of the edge pixel 630 may be generated, and in step S260, the edge pixel 630 may be classified as a pixel of the area of the face based on the edge-gradient orientation. On the other hand, the fourth edge pixel 640 may be classified as a pixel in the area of the hand in step S260.

Figure 7 illustrates a method of classifying an edge pixel according to an embodiment of the present invention into one of the pixels of the face area and one of the pixels of the hand area.

7, the face line 710 of the superimposed image and the face line 720 of the face template are shown. An edge pixel 711 on the face line 710 of the superimposed image is shown as the edge pixel of the first edge described above with reference to Figure 2 and an edge pixel 721 on the face line 720 of the face template Is shown as an edge pixel corresponding to an edge pixel 711 on the face line 710 of the superimposed image. The edge pixel 721 on the face line 720 of the face template is the edge pixel 711 closest to the edge pixel 711 on the face line 710 of the superimposed image among the one or more edge pixels of the face line 720 .

7, the edge gradient orientation 712 of the edge pixel 711 on the face line 710 of the superimposed image and the edge gradient orientation 722 of the edge pixel 721 on the face line 720 of the face template, Was shown. [Delta] [theta] may be the intercept angle between the edge gradient orientation 712 and the edge gradient orientation 722. [

In one embodiment of the present invention, the edge gradient orientation difference of the edge pixel 711 may be a value corresponding to the intercepted angle [Delta] [theta]. In other words, in the edge pixel of the first edge described above with reference to FIG. 2, the edge gradient orientation difference of the edge pixel of the first edge is determined by the edge gradient orientation of the edge pixel of the first edge of the superimposed image, And the edge-gradient orientation of the edge pixel of the second edge corresponding to the edge pixel of the first edge. For example, when the normalization for ?? is performed, the edge gradient orientation difference of the edge pixels may be a value obtained by dividing the inclined angle ?? of the edge pixels by? / 2.

As described above, according to the description of step S260 of FIG. 2, when there is no other edge of the face template in the predetermined area including the edge pixels of the superimposed image, the area determining unit 130 determines The value of the edge gradient orientation difference may not exist, but the value of the edge gradient orientation difference may be determined to be a predetermined value for convenience of operation. For example, the area determining unit 130 may set the intercepted angle [Delta] [theta] to the maximum for the edge pixels when the interleaved angle [Delta] [theta] and the edge gradient orientation difference can not be calculated because there is no other edge corresponding to the edge pixel of the superposed image / 2, and the edge gradient orientation difference can be determined as a maximum value (for example, 1, which is the maximum value when the normalization is applied to the interposed angle ??).

The interleaved angle of edge pixels and the edge gradient orientation difference may be defined for edge pixels or edge pixel points. For example, when the inclined angle of the point (x, y) is ?? (x, y), the edge gradient orientation difference? _Diff (x, y) at the point (x, y) Can be defined.

In the step S260 described above with reference to Fig. 2, the area determining unit 130 determines whether or not the edge pixels overlap the edge pixels based on the edge gradient orientation difference? _Diff (x, y) A pixel in the area of the face where the hand is removed and a pixel in the area of the overlapping hand. For example, the area determining unit 130 determines the edge pixels of the point (x, y) when the edge gradient orientation difference? _Diff (x, y) of the edge pixels is less than or equal to a predetermined value, (X, y) can be classified into pixels of a region of overlapping hands if the edge gradient orientation difference [theta] _diff (x, y) is larger than a predetermined value .

8 (a) to 8 (j) show drawings for explaining a method of classifying an edge pixel according to an embodiment of the present invention into one of a pixel of a face area and a pixel of a hand area.

8 (a) to 8 (j), the black line represents the edge of the face template, the sky blue line represents the face edge included in the superimposed image, the gray represents the edge of the hand contained in the superimposed image, Represents the normalized line segment.

8 (b), the edge pixel P (x, y) on the face edge of the superimposed image is superimposed on the face template edge (black) and the overlapped image (sky blue and gray) in drawing a line l ₁ in the normal direction, and Fig. 8 when a line segment is called Q l ₁ and the face that the template is met, as shown in (c), obtains a normal line segment l ₂ for Q.

Next, as shown in FIG. 8 (d), the angle ?? of the line segment l ₁ and the line segment l ₂ can be obtained by the following equation.

8 (a) to 8 (d) is performed for all the edges by the search window 3x3 (the size of the widow is not limited thereto) Respectively. For reference, the position of the search window is assumed on the assumption that the central portion of the search window moves on the sky blue line. However, the present invention is not limited to this, and it is assumed that at least one vertex of the search window moves on the sky blue line And the like.

Next, referring to FIG. 8 (f), a state in which the search window has moved to the hand edge portion is shown. In this way, until the search window moves to the hand edge portion, (that is, the edge pixel in the face template is displayed in black) only one present, while Figure 8 when the search window has reached the hand edge portion, as shown in (f), 8 (g), the line segment l ₁ face template edge, as shown in Point Q ₁ intersects with the edge of the search window, but the point R ₁ at which the line segment l ₁ meets the edge of the hand is not detected in the search window, so it is not regarded as the edge of the face. Here, as shown in Fig. 8 (h), when the line segment l ₃ of the normal component is drawn at R ₁ , which is the intersection of the hand edge, and the angle of the hand edge and the elongated component are found as shown in Fig. 8 (i) . Also, it can be seen that there are no other edge pixels searched in the search window centered on the point R ₁ , thus indicating that the gray area is the edge of the hand. On the other hand, if the above process is repeated, another point on the hand edge pixel can be found as shown in FIG. 8 (j).

Example of processing of separation of overlapping hands of the present invention

FIG. 9 illustrates an example of a result of separating a hand edge from a superimposed image according to an embodiment of the present invention.

As can be seen from Equation (3), when the edge gradient orientation difference ?? of the edge pixel is close to zero and? _Diff (x, y) is less than or equal to a predetermined value (X, y) is larger than a predetermined value (i.e., when the edge gradient orientation difference [theta] _diff (x, y) is larger than a predetermined value ), It can be classified into the edge pixels of the hand portion of the superimposed image. In FIG. 9, this portion is indicated by a solid line.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

100: Image processing device
110: Color processor
120:
130:

Claims (25)

CLAIMS 1. A method for separating a second object nested in a first object in an image,
(a) an image processing apparatus, comprising: a superimposed image in which the second object is an image in a state in which the second object is superimposed on the first object; and an image of the first object in a state in which the second object is not superimposed on the first object Identifying a first edge included in the superimposed image among the extracted edges in a state in which the template for the superimposed image is combined;
(b) the image processing apparatus judges whether or not at least one edge pixel of the second edge exists in a predetermined area including edge pixels of the first edge; And
(c) the image processing apparatus determines an edge pixel of the first edge as one of the pixels of the first object and the pixels of the second object based on whether at least one edge pixel of the second edge exists step
, ≪ / RTI &
In the step (c)
If edge pixels of the second edge are present
(c1) the image processing apparatus generating an edge gradient orientation of edge pixels of the first edge; And
(c2) the image processing apparatus determining an edge pixel of the first edge as one of the pixels of the first object and the pixels of the second object based on the edge gradient orientation,
In the step (c2)
Wherein the image processing apparatus is configured to calculate an edge pixel of the first edge based on a difference value between an edge gradient orientation of the edge pixel of the first edge and an edge gradient orientation of the edge pixel of the second edge, And determining one of the pixels of the second object. The method according to claim 1,
In the step (c)
Wherein the image processing apparatus determines an edge pixel of the first edge as a pixel of the second object if edge pixels of the second edge do not exist. delete delete The method according to claim 1,
In the step (c2)
Wherein the image processing apparatus classifies edge pixels of the first edge into pixels of the first object when the difference value is less than or equal to a predetermined value and if the difference value is larger than the predetermined value, The edge pixels of the second object are classified into pixels of the second object. The method according to claim 1,
In the step (c2)
Wherein the edge pixel of the second edge is the edge pixel of the template closest to the edge pixel of the first edge. 3. The method of claim 2,
After the step (c)
(d) the image processing apparatus further comprises: a part of the area of the first object in a state where the second object is superimposed on the first object, the part where the change between the template and the superimposed image satisfies a predetermined criterion Adding to the second object
≪ / RTI > The method according to claim 1,
Before the step (a)
(a0). The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus is configured to calculate the color of the first inspection area based on the difference between the colors of the first inspection area including the area of the first object and the colors of the second inspection area including the first inspection area Whether or not
≪ / RTI > 9. The method of claim 8,
Wherein the first inspection area is a polygon or a circle having a minimum size including the area of the first object. The method according to claim 1,
Before the step (a)
(a1) defining a color model corresponding to the first object and the second object
Further comprising:
Wherein the image processing device identifies an area of the first one of the images based on the color model. The method according to claim 1,
Wherein the color information of the first object and the color of the second object are similar. The method according to claim 1,
Wherein the first object is a user's face and the second object is a user's hand. 12. A computer program for executing the method according to any one of claims 1, 2 and 5 to 12. An image processing apparatus for separating a second object superimposed on a first object in an image,
Wherein the first object is an image in a state in which the second object is superimposed on the first object, and the second image is a superimposed image in a state in which the template for the image of the first object in a state in which the second object is not superimposed on the first object is combined An image processing unit that identifies a first edge included in the superimposed image of the edge and determines whether there is at least one edge pixel of the second edge within a predetermined area including edge pixels of the first edge; And
And determining an edge pixel of the first edge as one of a pixel of the first object and a pixel of the second object based on whether at least one edge pixel of the second edge exists,
, ≪ / RTI &
If edge pixels of the second edge are present,
Wherein the image processing unit comprises:
Generating an edge gradient orientation of the edge pixel of the first edge,
Wherein the area determining unit
Determining an edge pixel of the first edge as one of the pixels of the first object and the pixels of the second object based on the edge gradient orientation
Lt; / RTI >
Wherein the image processing unit comprises:
A difference value between an edge-gradient orientation of the edge pixel of the first edge and an edge-gradient orientation of the edge pixel of the second edge,
Wherein the area determining unit
Edge pixels of the first edge and pixels of the second object based on the difference between the edge-gradient orientation of the edge pixels of the first edge and the edge-gradient orientation of the edge pixels of the second edge, ≪ / RTI > 15. The method of claim 14,
Wherein the area determining unit
And determines an edge pixel of the first edge as a pixel of the second object if edge pixels of the second edge do not exist. delete delete 15. The method of claim 14,
Wherein the area determining unit
Classifying edge pixels of the first edge into pixels of the first object when the difference value is less than or equal to a predetermined value and outputting edge pixels of the first edge to the pixels of the first object when the difference value is less than a predetermined value, Lt; RTI ID = 0.0 > 2 < / RTI > objects. 15. The method of claim 14,
Wherein the edge pixel of the second edge is the edge pixel of the template closest to the edge pixel of the first edge. 16. The method of claim 15,
Wherein the area determining unit
Adding a portion of a region of the first object in a state where the second object is superimposed on the first object to a region of the second object in which a change between the template and the superimposed image satisfies a predetermined criterion
Lt; / RTI > 15. The method of claim 14,
Wherein the image processing unit comprises:
Determining whether the image corresponds to the superimposed image based on a difference between a hue of a first inspection area including an area of the first object and a hue of a second inspection area including the first inspection area
Lt; / RTI > 22. The method of claim 21,
Wherein the first inspection area is a polygon or a circle having a minimum size including the area of the first object. 15. The method of claim 14,
Further comprising a color processing section for defining a color model,
Wherein the image processing unit identifies an area of the first one of the images based on the color model. 15. The method of claim 14,
Wherein the color information of the first object and the color of the second object are similar. 15. The method of claim 14,
Wherein the first object is a user's face and the second object is a user's hand.

Images