KR101441333B1 - Detecting Apparatus of Human Component AND Method of the same - Google Patents

Abstract

An apparatus and method for detecting a human body of an input image are provided. The present invention relates to a method and system for analyzing a difference between an image of an exercise database storing a positive sample and a negative sample of a human part, an image processor calculating a difference image for the input image, A sub window processor for extracting a feature set from the difference image and a human partial classifier for detecting a human body part corresponding to the human body partial model using a human body partial model obtained by learning about a feature set.

Body part, detection, window

Description

TECHNICAL FIELD [0001] The present invention relates to a human body part detecting method and a human body part detecting method.

The present invention relates to an object detecting apparatus and method, and more particularly, to a human body detecting apparatus and method, an object detecting apparatus, and a method thereof using a difference image as a feature image.

Object detection is crucial for video frequency analysis techniques, such as content-based video or image object recovery techniques, video surveillance, object recognition, frequency control, video image compression, and automated operator assistance. In the real world, human detection technology is one of the areas of intense competition. There are three situations in which a person is detected.

The first is a detection technique that determines whether a person is present in the field. For example, when the object detection system assists the driver, the driver is alerted to the person detection signal when the driver of the road approaches the vehicle. Such an object detection system can be realized with an embedded smart device integrated in an image forming apparatus.

The second situation is the sequential tracking of the same person used in a still or PTZ camera. The still camera obtains a moving trajectory of a person and applies it to smart behavior analysis. The PTZ camera adjusts the camera state to the person moving and records details and actions with it as the center of the image.

The third situation is when robots want to track people, watch people, or interact. The robot wants to detect the position of a person in the image of the camera and performs a corresponding action, for example, movement, tracking, viewing or status adjustment. This can be realized with the entire smart device integrated in the robot's functional unit.

Due to the various forms and styles of fashion, it is relatively difficult to find features that fall within a particular category, as there are various variations in the overall shape as well as the partial appearance of the person. Therefore, a feature set with stability and discrimination is required even in complicated backgrounds under difficult lighting illumination. In addition, the overall shape can deform a person's silhouette due to the problem of accuracy, deformation due to many obstacles, or the appearance of a large number of people in the same image area. Therefore, there is a need for an algorithm that can overcome small interferences among the entire recorded images and deduce accurate results.

Conventional human detection methods use a method that detects an entire model, e.g., an entire human shape or contour, or uses a partial feature set or a partial detector. The detection method using the entire model extracts the characteristics of the entire human body to form an overall model according to the appearance or silhouette. Detection methods using partial detectors divide the human body into several components, such as head, body, leg, and arm. The detection technique for such a person can be referred to as a partial detection technique, and the research on each part is focused on simplifying the human part model. Pattern learning methods that simplify partial models include SVM, adaboosting, and other methods.

Especially, face detection technology of man has been developed in recent years, and it can be seen that high detection rate and low error are realized when real-time detection technology is executed. However, in practical applications, human detection still requires a lot of work. First, a human detector should be applied in accordance with changes in a person's appearance as a result of costume design and changes in different lighting conditions. Second, the human detector should be made on the basis of a stable feature that compensates for the feature despite various variations of human appearance. Finally, the human detector requires a relatively small amount of computation and real-time processing.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide an apparatus and method for detecting a human body having a stable identification characteristic and a real-time processing capability.

It is still another object of the present invention to provide an object detecting apparatus and a method therefor which have a stable identification characteristic and a real-time processing capability.

It is another object of the present invention to provide an image forming system having stable identification characteristics and real-time processing capability.

In order to solve the above-described problems,

A training database for storing positive and negative samples of the human body; an image processor for calculating a difference image for the input image; and an image processor for calculating positive and negative samples of a predetermined human body portion stored in the practice database A sub-window processor for extracting a feature set from the difference image, and a human partial classifier for detecting a human body portion corresponding to the human body sub-model using the human sub- A human body part detecting device is provided.

In order to solve the above-mentioned problems,

The method comprising the steps of: storing a positive sample and a negative sample of a human body part and extracting a feature set from the difference image of the positive sample and the negative sample; learning the extracted feature set to obtain a human body partial model; And detecting the corresponding human part of the difference image using the human body partial model.

In order to solve the above-mentioned problems,

A training database for storing positive and negative samples of an object; an image processor for calculating a difference image for the input image; and a difference image computed by the image processor for positive and negative samples of the object stored in the practice database A sub-window processor for extracting a feature set and an object classifier for detecting an object corresponding to the object model using an object model obtained by learning about the feature set.

In order to solve the above-mentioned problems,

The method comprising the steps of: storing a positive sample and a negative sample of an object and extracting a feature set from the difference image of the positive sample and the negative sample; learning the extracted feature set to obtain a target model; Calculating an image, and detecting a corresponding one of the difference images using the object model.

In order to solve the above-mentioned problems,

And an object model obtained by performing learning on a feature set extracted from a difference image of the positive sample and the negative sample of the object with respect to the difference image of the object, A state variable calculation device for calculating a state variable based on an object area photographed by the imaging object detected by the detection device and positioning the object in a central area of the image, A control device for receiving the state variable from the state variable calculation device and adjusting the state, a storage device for storing the image of the object, and a display device for displaying the image of the object do.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a view showing a structure of a head part detecting device of a human body part detecting device according to an embodiment of the present invention.

1, the head detecting apparatus 100 includes an image processor 110, an exercise database 120, a sub window processor 130, and a head classifier 140 .

The image processor 110 calculates the difference image of all input images. The difference image is calculated not only for the images input by the camera in real time but also for the sample images stored in the practice DB.

The practice DB 120 stores a positive sample and a negative sample of the human body portion, wherein the human body portion is a human head portion, so that the positive sample and the negative sample are captured with respect to the human head .

The sub window processor 130 extracts a feature set for the difference image using the difference image calculated by the image processor 110 for the positive sample and the negative sample of the head of the person stored in the practice DB 120. [

The head classifier 140 proceeds to learn about the extracted feature set to obtain the head model and then sends the image processor 110 a new difference image calculated by the image processor 110 for the input image, And detects the head area.

As can be readily appreciated by those skilled in the art, the head-portion detecting device 100 shown in Fig. 1 is merely for illustrative purposes, and the head classifying device 140 of the head-portion detecting device 100 is replaced by a sorter of another human body For example, a body sorter, a leg sorter, an arm sorter, and an entire sorter, thereby forming a detection device for other human body parts, thereby detecting other human body parts in the image. In addition, the head classifier 140 of FIG. 1 may be configured in the form of a pyramid classifier 800 shown in FIG. 9 for a head that may appear at various imaging angles shown in FIG.

In this way, any human body part detecting device can be configured including an image processor, a practice DB, a subwindow processor, and a human partial classifier. In this case, each of the human body detecting apparatuses may include all of the four elements, or may include an image processor, a practice DB, and a subwindow processor. The human body partial classifier may be separately provided for each human body.

FIG. 2 is a diagram illustrating a method of calculating an image difference by an image processor according to an embodiment of the present invention. FIG.

Referring to Fig. 2, the image processor 110 calculates four difference images of the first through fourth difference images dx, dy, du and dv. Each pixel value of the first difference image (dx) is the image difference in the horizontal direction among the N * N adjacent regions of the original image. Referring to the first difference image (dx) of FIG. 2, if N is 3, the sum of pixel values of gray circles is subtracted from the sum of pixel values of gray rectangles to obtain the center pixel value of the first difference image (dx) do.

Each pixel value of the second difference image (dy) is the pixel difference value in the vertical direction in the N * N adjacent region of the original image. Referring to the second difference image dy in FIG. 2, when N is 3, the sum of the pixel values of the gray rectangle is subtracted from the sum of the pixel values of the gray rectangle to obtain the pixel value of the center of the second difference image (dy) .

Each pixel value of the third difference image du is a pixel difference value in the left-right diagonal direction among the adjacent regions of the original image N * N. Referring to the third difference image du in FIG. 1, when N is 3, the sum of the pixel values of the gray rectangle is subtracted from the sum of the pixel values of the gray rectangle to obtain the pixel value of the center of the third difference image du .

Each pixel value of the fourth difference image (dv) is a pixel difference value in the left-right diagonal direction among N * N adjacent regions of the original image. Referring to the fourth difference image dv in FIG. 2, when N is 3, the sum of the pixel values of the gray rectangles is subtracted from the sum of the pixel values of the gray circles to obtain the center pixel value of the fourth difference image dv. In this way, each pixel value of the first to fourth difference images (dx, dy, du, dv) represents the change in the average gray level of the pixels in the adjacent region in the target direction.

In addition, difference images can be calculated at different scales. For example, if the sum of the pixel values of the black square among the first to fourth difference images dx, dy, du, and dv of FIG. 1 is subtracted from the sum of the pixel values of the black circle, Pixel value. (2 * n + 1) * (2 * n + 1), n = 1,2 ,,,,,, can be expressed as the center pixel value of the adjacent region.

The image is sub-sampled for different scales, for example, if n is 2, the difference image calculation is performed once for each pixel. In order to obtain a low resolution image, a relatively large scale, that is, a relatively large adjacent region may be set to calculate the difference image, thereby reducing the influence of background noise in extracting the feature set. Further, in order to obtain a high resolution image, a pixel value can be displayed in a finer manner by calculating a difference image by setting a relatively small scale, that is, a relatively small adjacent region.

For example, a difference image is calculated with a width of 1 to w-2 and a height of 1 to h-2 for a pixel whose image width is w and h height, and a pixel value outside the image boundary in calculation is set to 0 do. Sub-sampling is performed on the original gray-scale image before calculating the difference image on a large scale. Then, the feature difference image is calculated according to the above description.

Figure 3 shows a typical negative sample having an outline shape similar to the outline of the positive sample of the head.

Referring to FIG. 3, a negative sample is similar to a rectangular line structure and appears as a modified form of a positive sample at the head of a person. From this, it is possible to test the discriminating ability of the classifier, since the error in the detection result generally includes the object having a shape similar to that of the human head. On the other hand, this shows the importance of feature transformation and extraction. The present invention reduces the difficulty of such feature extraction.

FIG. 4 shows a difference image of an object having a thick straight line.

Referring to FIG. 4, an object having a linear structure among images may be classified into different difference images. This distinction provides a relatively good source for feature extraction. The first difference image dx maintains the horizontal variation of the image and the second difference image dy maintains the vertical variation of the image. The third and fourth difference images du and dv retain the diagonal variation of the image.

Fig. 5 shows four examples (dx, dy, du, dv) of the difference image of the head calculated according to the method shown in Fig. Compared with the difference image of FIG. 4, the straight line is decomposed in the difference image of FIG. 3, but the contour of the head portion is relatively well preserved.

FIG. 6 shows a sub-window used in a sub-window processor among feature extraction methods according to an embodiment of the present invention.

First, a single window is created by changing the width and height of the training image at a rate that can be adjusted according to the image ratio, thereby generating the characteristic of the single window. Then, on the practice image, the width and the height are changed to the scale according to the scale adjustment of the image, and at the same time, the width and the height of the two windows are changed with the same zoom variable to capture a hidden pattern having a wide range according to the scale To form two window features by sliding two windows. The two windows move relative to each other in the horizontal and vertical directions to capture a wide range of hidden patterns.

Slide three connected windows on the exercise sample to form three window features. The three windows have the same scale and height in the scale change to capture a wide range of models according to the scale. The second window captures the contours of the convex and concave for the movement of the first window and the third window. There are two types of window features: one is the horizontal arrangement of three windows, and the other is the vertical arrangement of the three windows. With horizontal placement, the second window horizontally moves relative to the first window and the third window; As a vertical arrangement, the second window moves vertically relative to the first window and the third window.

Let f be the extracted feature, G be the difference image, w be the feature window, and let OP ¹ be the first operator used for feature extraction. OP ¹ Is a first operator containing two simple operators '+' and '-', ie, OP ¹ = {+. -}to be. OP ² is a second operator used in the present invention, including '+', '-' and a main operator (domin), and OP ² = {+. -, domin}.

In order to extract the feature of one difference image, the feature can be calculated by Equations 1, 2, and 3 corresponding to a single feature window, two feature windows, and three feature windows, respectively. a can be one of the four difference images.

For the feature extraction of the two difference images, the feature can again be calculated by equations (4), (5) and (6) corresponding to the single feature window, the two feature windows and the three feature windows of the difference image. a and b are any two images of the four difference images.

For the feature extraction of the three difference images, the feature can be calculated through equations (7), (8) and (9) corresponding to a single feature window, two feature windows and three feature windows, respectively. a, b, and c are any three of the four difference images.

For the feature extraction of the four difference images, the feature is calculated through equations 10, 11, 12 corresponding to a single feature window, two feature windows and three feature windows, respectively. a, b, c, and d are four difference images.

As shown in Equations (13) to (17), the first operator (OP ¹ ) is an operator of addition and subtraction, and the second operator (OP ² ) includes a main operator as well as addition and subtraction operators.

Although FIG. 6 illustrates the sub-window used in the feature extraction method, it is not intended to limit the object of the invention. The present invention is feasible with sub-windows of various numbers and types.

Sub-window processor 130 extracts features by scanning one difference image, two difference images, three difference images, or four difference images. A feature set may include one difference image, two difference images, three difference images, or one feature of a feature extracted from four difference images, or one difference image, two difference images, three difference images, Lt; RTI ID = 0.0 > a < / RTI > In addition, different images can be computed using different scales to obtain more features, and feature sets can be extracted from difference images computed on different scales.

A final classification model can be generated by selecting a feature having discrimination capability using a statistical learning method, for example, Adaboost, SVM, or the like for the extracted feature set. Among the various model identification technologies, the model classification technique can be applied to a model classifier.

7 is a flowchart briefly showing a method of detecting a human body part according to an embodiment of the present invention.

Referring to FIG. 7, in the human body part detecting method according to the embodiment of the present invention, first, a positive sample and a negative sample of a human body part are prepared, a difference image between the positive sample and a negative sample is calculated, A feature set is formed using the feature extraction method (S1)

A partial model of the human body part is formed using the learning method using the statistical method (S2). By obtaining such a partial model, a human partial classifier using the model can be obtained. For example, when a positive sample and a negative sample of the body are prepared for the body, a feature set is extracted using the feature extraction method described above for the difference image between the positive sample and the negative sample of the body. Then, statistical learning methods are used to select the identifiable features and eventually form a body model and a body classifier that encodes it.

The corresponding human body part of the difference image of the input image is detected using the detected human body model (S3).

Fig. 8 is a view showing the division of the human head in various viewing angles.

8, when the angle at which the camera illuminates the head changes, the image of the head portion is displayed in eight directions, that is, front, left half, left, left rear half, right half, It is classified as the posterior side of the posterior margin. Eight separate images surround the head and display all images at an angle of 360 degrees. Each image includes a range of angles and is not limited to a specific angle. For example, if the front image is 0 degrees and the left side is 90 degrees, the front image is actually [-22.5, +22.5], the left half image is actually [22.5, 67.5] .

9 is a view showing a pyramid classifier for detecting a human head.

The pyramid classifier 800 shown in FIG. 9 receives the feature set of the difference image for the positive sample and the negative sample with respect to the eight parts of the head shown in FIG. 8, and encodes the head part model corresponding to each feature set And first to fourth classifiers A, F, P and HP for performing partial detection on the input image.

In other words, the pyramid classifier 800 can learn the feature using the feature set extracted from the difference images of the positive and negative samples on the front, left half, left half, left half, rear half, right half, A first head classifier (A) using the first head model and the first head model obtained, and a second head classifier A second head classifier (F) that uses the second head model and the second head class model obtained by the second head classifier; and a second head classifier A third head classifier (P) using the third head model and a third head classifier (P) using the third head model and the second head classifier Using the feature set extracted from the difference image of the sample and a negative sample and a fourth head classifier (HP) with a fourth head and the fourth head model model learning is obtained.

The image processor (not shown) at the time of detection first calculates the difference images of the four input images in the horizontal, vertical, left-right diagonal, and right-left diagonal directions, And the first to fourth classifiers sequentially search for and measure each possible scale and position of the difference image. The evaluation of the pyramid detector 800 is accepted as a candidate for the human head. Otherwise it is discarded as an error.

In detail, the first classifier A first searches for and measures each possible scale and position in the image. That is, the first classifier A detects the head area, and the head area detected by the first classifier A is evaluated by the second to fourth classifiers F, P, and HP. If the sample is received by the second to fourth classifiers F, P and HP, the head sample is determined as a candidate for the head part image corresponding to each of the second to fourth classifiers F, P and HP do. Each of the second to fourth classifiers F, P, and HP progressively evaluates the samples received by the first classifier A to complete the evaluation of all the samples received by the first classifier A .

FIG. 10 is a flowchart illustrating the detection of the pyramid classification 800 shown in FIG. 9 for explaining a human body part detection method according to an embodiment of the present invention.

Referring to FIG. 10, first, positive and negative samples of the front, left half, left, rear half, rear, right half, right half and right half of the human head are prepared. The features are then extracted from the difference images of the positive and negative samples on the front, left half, left, left half, rear, right half, right half, and right half of the human head.

Learning is performed on the feature set of the full angle of the head to acquire the first head model and a feature set extracted from the difference image between the positive sample and the negative sample on the front and rear sides of the head is learned, Get a model. Further, learning is performed on the feature set extracted from the difference image between the positive and negative samples on the left and right sides of the head to acquire the third head model, and the left half side, the left half side, the right half side, Learning is performed on the feature set extracted from the difference image between the positive sample and the negative sample on the right half side to obtain the fourth head model.

The first to fourth classifiers A, F, P and HP of the pyramid classifier 800 use the models of the first, second, third and fourth heads, respectively. During the detection process, first, the difference image of the input image is calculated in four difference images: horizontal, vertical, left-right diagonal, and right-left diagonal. The pyramid classifier 800 detects an input image and transmits the detected human head portion as a detection result.

According to another aspect of the present invention, there is provided an apparatus for detecting a human body, comprising: at least one human body part other than a human head, for example, a classifier of a human body, a leg part and an arm part, It is possible to improve the detection precision of the human head by eliminating the error of the human head determined from the pyramid classifier 800 shown in Fig. Further, the human head detecting apparatus according to the embodiment of the present invention is used as a detecting apparatus for a human being, because it can determine the existence of a person when a human head is detected.

FIG. 11 shows a configuration of a person detecting apparatus 1000 in which a plurality of human body detecting apparatuses are arranged in a line according to an embodiment of the present invention.

Referring to Fig. 11, the human detection device 1000 includes N partial detection devices arranged in a row, and one of them is a main detection device. The human detection device 1000 is used for detection of a human body portion corresponding to the main detection device among input images. In the embodiment of the present invention, the first part detecting device I is used as the main detecting device.

The operation of the person detecting apparatus 1000 will be described. First, the detected input image is input to the first partial detection device I. The first part detecting device I detects an area corresponding to the first human part I of the input image.

Then, the second partial detecting device II detects an area corresponding to the second human body part II of the input image. At this time, based on the geometric relationship of the human body, based on the area of the second human body part II detected by the second part detecting device II, the first human body part I Quot; candidate " candidates is rejected as an error, and a part of the candidates of the second human part (II) is erroneously processed to leave the second human part (II).

In this way, the other partial detectors (not shown) remove part of the candidate that does not correspond to the corresponding human body portion of the input image as an error and transmit the remaining human body portion (not shown) to the next partial detector.

The Nth partial detector N detects an area corresponding to the Nth human part N among the input images. At this time, on the basis of the area of the Nth human body part N detected by the Nth partial detector N, the position of the first human body part I evaluated by the second partial detector II Removes a part of the partial candidates as an error, and removes a candidate part of the second to N-1th parts (II to N-1) and the candidate part of the Nth part (N) as an error. The second human body part II, the third to the N-1th human body parts III to N-1 and the Nth human body part N corresponding to the remaining part of the first human body part I are obtained have.

In comparison with the detection using the detector of the first human body I in this way, detection with respect to the first human body I can be progressed using multiple detectors in combination to obtain accurate results. Further, in this manner, a plurality of partial detecting apparatuses can be used to detect among the input images according to human body geometry and other human body parts.

The second part detecting device II operates as a verifier from the partial candidate of the first partial detector I. In the above-described situation, the second partial detection device II detects the second human part II by scanning the proximity area from the partial candidate of the first partial detection device I. The adjacent area is determined according to the human geometry relationship. For example, if the first partial detector I is a head detector, then the second partial detector II is a body detector, but the correct head candidate of the head part candidates of the first partial detector I is the relative position of the body The position and scale of the body of the input image to be detected are determined roughly according to the relationship and scale relationship, and statistical analysis is performed.

In the present invention, the partial detecting apparatus of the person detecting apparatus 1000 is not limited to a specific number, and a partial detecting apparatus may be constituted according to a necessary portion to implement the person detecting apparatus 1000. [ The partial detecting device may be a head detecting device, a body detecting device, a leg detecting device, an arm detecting device, and a whole human body detecting device.

12 is an exemplary detailed structural view of a person detecting apparatus 1100 having a plurality of human body part detectors of FIG. 11 according to an embodiment of the present invention.

12, the human detection apparatus 1100 includes a determiner 1110, a head detecting apparatus 1120, a leg detecting apparatus 1130, and a body detecting apparatus 1140. [ In the embodiment, the head detecting device 1120 may be the main detecting device and the head detecting device 100 shown in FIG. The leg detecting device 1130 and the body detecting device 1140 have the same structure as the head detecting device of FIG.

The head detecting apparatus 1120, the leg detecting apparatus 1130 and the body detecting apparatus 1140 may include an image processor (not shown), a practice DB (not shown), and a sub window processor (not shown) The respective devices have the same functions as the image processor 110, the practice DB 120, and the sub-window processor 130 of FIG. 1, and therefore, detailed description thereof will be omitted.

In addition, the head detecting apparatus 1120 includes a head classifier obtained from a learned head model from a feature set, and the sub window processor of the head detecting apparatus 1120 includes a training DB of the head detector 1120 And a difference image between the positive head sample and the negative sample calculated in the image processor of the head detecting apparatus 1120. The difference image between the head sample and the negative sample,

The leg detecting apparatus 1130 includes a leg classifier obtained from a leg model obtained by learning from a feature set, and the sub window processor of the leg detecting apparatus 1130 includes a practice DB of the leg detecting apparatus 1130 Includes a feature set extracted from a difference image of a positive sample and a negative sample of a leg portion stored in the leg portion detecting unit 1130 and includes a difference image of a head positive sample and a negative sample calculated in the image processor of the leg detecting apparatus 1130.

The body detection apparatus 1140 also includes a body classifier obtained from the body model learned from the feature set, wherein the sub-window processor of the body detection apparatus 1140 detects the positive And a difference image between the positive body sample and the negative sample calculated by the image processor of the body detection apparatus 1140. [

First, the head detecting apparatus 1120, the leg detecting apparatus 1130, and the body detecting apparatus 1140 detect an image, respectively, and transmit head candidates, leg candidates, and body candidates.

The determiner 1110 detects a human body geometry from the head candidate of the head detecting apparatus 1120 and the body candidate of the body detecting apparatus 1140 according to a relative constraint and a scale constraint of the human head and body Eliminate errors from the head candidate. Then, the determiner 1110 removes the error-warped head candidate and the bridge candidate according to the combined body candidate, further removes the error of the head candidate according to the human body geometry, And the human body.

The head detecting apparatus 1120 shown in Fig. 12 may also be a form including the pyramid classifier 800 shown in Fig. That is, the head detecting apparatus 1120, the leg detecting apparatus 1130 and the body detecting apparatus 1140 are respectively connected to the image processor, the practice DB, the sub window processor, the body detector 120, the leg detector 130, A practice image DB, a subwindow processor without using the detector 1140. In the example of FIG.

13 is a structural diagram of an object detecting apparatus 1200 to which a plurality of partial detecting apparatuses according to another embodiment of the present invention are connected in parallel.

Referring to Fig. 13, the object detection apparatus 1200 includes n partial detectors I-n arranged in parallel. The object detection apparatus 1200 may also include an image processor (not shown), a practice DB (not shown) and a subwindow processor (not shown) ) And the sub window processor 130, detailed description thereof will be omitted. Each of the plurality of first to Nth partial detectors IN may have an image processor, an exercise DB, and a sub-window processor, such as the object detection apparatus 1100 of FIG. 12, or may be a pyramid classifier 800 of FIG. , A common image processor, a practice DB, and a subwindow processor.

In addition, the object detecting apparatus 1200 may further include a determinator (not shown). When the object to be detected is a human, the object detecting apparatus 1200 may detect a human body part detected by the first to n- The human body part and the person in the detected image are determined based on the error in the human body part.

A first detection unit (I) is m ₁ of the first portion of the classifier (S11, S12, S13, ... S1 m ₁₎ and including a second detecting device (II) is the ₂ m out of the second portion classifier (S21, S22, S23, ... S1 m ₂₎ comprises a n-th detection apparatus (n) (n = 1,2,3, ... n is m _n of the n-classifier (S21 of the portion , S22, S23, ..., S1m _n (where m1, m2, ... mn are natural numbers).

Since the method of executing the classifier and the method of the human body detecting apparatus shown in FIG. 12 are similar to each other, the description of the execution of the classifier will be omitted. As described in FIG. 13, the classifiers of the n-part detecting apparatuses (I, II, sn) of the object detecting apparatus 1200 execute necessary calculations in ascending order. In general, since the amount of computation of a classifier corresponds to the amount of computation for processing a number of features, the classifier arranges the ascending order of the number of features of the feature set.

Hereinafter, the operation of the object detection apparatus 1200 will be described with reference to FIG.

After the image is input to the object detection apparatus 1200, an image is detected first through two front classifiers (i.e., S11 and S12) in the first detection apparatus I, that is, To obtain a candidate for the first part of the device (I). Then, the candidate of the first part is transmitted to the second detecting device II, that is, it is driven to the point B, and then the image is detected through the two front end classifiers (i.e., S21 and S22) To obtain a candidate of the second part of the second detecting device II. Then, the determiner (not shown) removes the error among the candidates of the first part according to the structural geometry of the object based on the candidate of the first part and the candidate of the second part obtained (proceeds to point D).

In this manner, the n < th > detector (n) is used for the determined portion in the previous stage (going to point E) (S 1, Sn 2, Sn 3,...) Of the first and second sheaves of the first and second sheaves are operated to obtain a candidate for the nth portion (proceeds to point F) Then, the n-th detection device n transmits to the first detection device (operates up to K) with respect to the remaining partial candidates (up to H), and again uses the classifier of the first detection device I And eliminates the error through the determiner. The third to nth detection devices (n) to n (n) are repeatedly used in accordance with the following sequence, and all of the classifiers of the first to nth detectors In are repeatedly used use.

The principle of this detection method is as follows. First, most error warnings are removed with a relatively small amount of computation by using a classifier of a front end classifier of each detection apparatus, that is, a classifier having a relatively small number of features of a feature set. Then, , And is based on the principle of rapidly improving the detection speed. If only the first detection device I is used for the same error, only three front classifiers S11, S12, S13 are required and all 50 features are removed, Only one of the five features can be erroneously removed by using only one classifier S21 of the second detection device II and S11 and S12 of the second detection device II.

Is automatically switched from one detection device to another according to the expected threshold value of the number of feature sets possessed by the classifier. Although the threshold values can be selected in different ways, the principle is similar, and if the characteristics of the feature set are relatively small in the classifier, the error can be removed by switching between different detection devices.

All the classifiers of the object detection apparatus 1200 of the embodiment of the present invention proceed by using each classifier in ascending order of the number of features existing in the feature set. When the detection is performed using the object detection apparatus 1200, considering the ascending order of the number of features of the feature set possessed by the classifier without considering the partial detection apparatus to which the classifier belongs, Each classifier in the apparatus 1200 is used.

The present invention can be applied not only to detection of a human body part and a person but also to detection of objects having a certain shape (for example, animals, plants, buildings, natural scenery and daily living goods).

FIG. 14 is a diagram showing the structure of an image forming system 1300 having an object detection function according to an embodiment of the present invention.

14, the image forming apparatus 1300 includes a photographing apparatus 1301, a detecting apparatus 1302, a variable calculating apparatus 1303, a control apparatus 1304, a storage apparatus 1305, a display apparatus 1306, And a notation device 1307. The image forming system may be one of a PTZ (PAN, TILT, ZOOM) camera, a stationary surveillance camera, a DC (digital camera), a camera mobile phone, a DV (digital video) and a PVR (personal video recorder).

The photographing apparatus 1301 is a hardware apparatus, for example, a CCD or a COMS apparatus, and is used for sensing and generating images of natural systems.

In order to track the subject, two types of methods are used to provide the subject's range of motion, position, and scale. The first method is an automatic method that extracts the area scales and locations of interest embedded in the object detection function. The second method is a manual method in which the user or operator can mark the area of interest on the displayed image (e.g., a touch screen). In the case of the automatic method, the object can be automatically detected using the object method according to the present invention. The notation device 1307 provides notational functions to a user or an operator so that a user or an operator can manually mark an image of interest with a pen or a finger.

The detecting device 1302 can receive image data from the photographing device 1301 and can receive the scale and positional information of a marked interested area in the form of, for example, a roughly indicated mark. The variable calculation device 1303 calculates a variable for adjusting the state of the image forming apparatus in accordance with the existence area of the object provided by the variable calculation device 1303, do. Note that marking device 1307 may also be selected when using the first method of providing the object location and scale, i.e., the automatic method. When a plurality of tracking objects are selected, for example, when tracking is performed for a plurality of objects, the user can modify the object to be tracked which is automatically selected by the image forming apparatus of the present invention.

15 is a diagram showing the structure of the detecting apparatus 1302 of FIG. 14 according to the embodiment of the present invention.

15, the object detection apparatus 1302 includes an image processor 1410, a practice DB 1420, a subwindow processor 1430, and a target processor 1440.

The image processor 1410 calculates the difference image of the image output by the photographing apparatus 1301 of Fig. 14 using the method described in Fig. The practice DB 1420 stores positive samples and negative samples of various objects (e.g., various kinds of creatures, plants, buildings, natural scenery, and daily living goods). The sub window processor 1430 extracts features from the difference image of the positive and negative samples of the object stored in the practice DB 1420, calculated from the image processor 1410 using the feature extraction method described in FIG.

The target classifier 1440 detects the target region of the object using the learned target model from the extracted feature set based on the difference image of the input image that is calculated by the image processor 1410. The above-described target model is obtained in the same manner as the learning method of the head model.

The object detection apparatus 1302 may further include a transmission apparatus (not shown). The transmission device transmits the target image detected by the target classifier 1440 or the target image directly selected by the user in the notation device 1307.

The detection device 1302 may not include the practice DB 1420 and the sub window processor 1430 and may pre-position the classification models of the various targets, that is, the target models in the target classifier 1440. The target classifier 1440 adjusts the target classification model according to the target type desired by the user.

The control device 1304 can adjust the state of the image forming apparatus and the state of the image forming apparatus can be controlled by the operation of the rotation, tilt, zoom and selective focus zone of the PTZ camera or the zoom of the stop monitoring camera, DC, DV or PVR And focus operation. The control device 1304 receives the state adjustment parameter of the image forming apparatus in the variable calculation device 1303. [

Detection device 1302 may provide position and scale information of the object in a new time point or new frame data. The control device 1304 controls the state of the image forming apparatus according to the variable to position the object at the center of the image by the rotation and tilting operation, to select the region of interest through manipulation of the selected focus region, To focus on the area of interest, so that the details can be obtained on the exercise object having high image quality.

During the operation of the selective focus area, the control device 1304 controls the image forming device to select a new area on which the object exists, on the basis of the focus, thereby focusing on the area. In addition, when the photographing apparatus 1301 selects the focus area, the control device 1304 controls not only to set the image center area to the acquired image focus area, but also to select a new image area in which the object is located as the image focus area And a better image effect can be obtained by dynamically adjusting the zoom factor, the focal length, the rotation or the tilt variable based on the image data information of the focus area.

For electronic products, such as DC, DV or PVR, that the user carries, the user can manually center the state of interest by adjusting its state. The control device of one embodiment of the present invention can dynamically adjust the zoom factor and focus variable of the photographing device based on the detection result and the detection result and the variable provided by the variable calculation device.

The storage device 1305 stores images or frequencies, and the display device 1306 displays images or frequencies of the scene to the user.

The detection device 1302 according to the embodiment of the present invention can be realized by software, which is applied as an embedded system to an image forming apparatus and a control apparatus, and becomes a variable for adjusting the state of the image forming apparatus. In the embedded image forming system, the state of the photographing apparatus, the mirror focus region, and the like can be adjusted by receiving the frequency as an input and transmitting the command to the control apparatus of the image forming apparatus.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Modification is possible. Accordingly, the spirit of the present invention should be understood only in accordance with the following claims, and all equivalents or equivalent variations thereof are included in the scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a view showing a structure of a head part detecting device of a human body part detecting device according to an embodiment of the present invention;

Figure 2 is a diagram illustrating a method for an image processor to calculate a difference image according to an embodiment of the present invention;

Figure 3 shows a typical negative sample having an outline shape similar to the contour of the positive sample of the head,

4 is a view showing a difference image of an object having a thick straight line,

FIG. 5 is a view showing four examples (dx, dy, du, dv) of the head difference image calculated according to the method shown in FIG. 2;

FIG. 6 is a diagram illustrating a sub-window used in a sub-window processor among feature extraction methods according to an exemplary embodiment of the present invention;

FIG. 7 is a flowchart schematically illustrating a human body part detection method according to an embodiment of the present invention,

FIG. 8 is a view showing the division of the human head in various viewing angles, FIG.

Figure 9 shows a pyramid sorter for detecting a human head;

10 is a flowchart showing an example of the detection of the pyramid classification 800 shown in FIG. 9 for explaining the human body part detection method according to the embodiment of the present invention,

11 is a configuration diagram of a human detection apparatus in which a plurality of human body part detection devices are arranged in a line according to an embodiment of the present invention;

FIG. 12 is an exemplary detailed structural view of a human detection device having a plurality of human body part detectors of FIG. 11 according to an embodiment of the present invention;

13 is a structural view of an object detecting apparatus to which a plurality of partial detecting apparatuses according to another embodiment of the present invention are connected in parallel,

14 is a diagram illustrating the structure of an image forming system 1300 having an object detection function according to an embodiment of the present invention.

15 is a diagram showing the structure of the detection apparatus of FIG. 14 according to the embodiment of the present invention.

Claims (30)

An exercise database storing positive and negative samples of the human body; An image processor for calculating a difference image for an input image; A sub-window processor for extracting a feature set from a difference image calculated by the image processor for positive and negative samples of a predetermined human body portion stored in the practice database; And A human partial classifier for detecting a human body portion corresponding to the human body partial model using the human body partial model for the feature set; Lt; / RTI > Wherein the human partial classifier comprises: Wherein the human body model is obtained by learning the feature set of the difference image between the positive sample and the negative sample with respect to at least one pair of side faces of the plurality of side faces of the human body part, A partial detection device. The method according to claim 1, Further comprising a determiner for determining a human body portion of the input image by removing an error based on a human body geometry with respect to the human body portion detected by the human body partial classifier. The method according to claim 1, Wherein the image processor calculates the input image as a difference image in a horizontal direction, a vertical direction, a left-right diagonal direction, and a right-left diagonal direction with at least one scale. The method of claim 3, Wherein the subwindow processor extracts the feature set from at least one of the difference images in the horizontal direction, the vertical direction, the left-right diagonal direction and the right-left diagonal direction using a single window or a plurality of windows. Human body part detection device. The method according to claim 1, Wherein the human body part is a head part, a body part, a leg part, an arm part, or any part of a human body. The method according to claim 1, Wherein the practice database includes a positive sample and a negative sample on the front side, left side, left side, left rear side, rear side, rear side, right side and right side of the head as positive and negative samples of the head when the human body is the head And stores the image of the human body in the image. The method according to claim 6, Wherein the human partial classifier comprises: Based on the difference image of the head portion, learning is performed on the feature set of the difference image between the positive sample and the negative sample on the front, left half, left, rear half, rear, right half, right half, A first head classifier for detecting the head using a first head model; A second head model obtained by learning about a feature set of a difference image between a positive sample and a negative sample on the front and back sides of the head based on the difference image of the head, A second head classifier for detecting the head; The left and right sides of the head are detected using the third head model obtained by learning the feature set of the difference image between the left and right positive and negative samples of the head based on the difference image of the head A third head classifier; And Using the fourth head model obtained by learning the feature set of the difference image between the positive sample and the negative sample on the left rear half side, the left half side, the right half side and the right half side of the head part based on the difference image of the head part A fourth head classifier for detecting the left half side, the left half side, the right half side and the right half side of the head part; And a display unit for displaying an image of the human body. 8. The method of claim 7, And the second to fourth head classifiers perform evaluation on the head portion detected through the first head classifier to detect an error on the front and rear surfaces of the head portion, an error on the left and right sides of the head portion, The left half, the right half, and the right half of the image of the human body. The method according to claim 1, Wherein the human partial classifier comprises a plurality of human partial classifiers according to a plurality of different human parts, and wherein the human partial model classifies the corresponding human parts in ascending order of the number of features of the human body partial model obtained by learning to correspond to the human part And the detection of the human body part of the image is progressed. 3. The method of claim 2, Wherein the human partial classifier performs detection for a corresponding human body part in ascending order of the number of features of the feature set of the human body partial model. Storing a positive sample and a negative sample of a human body portion and extracting a feature set from the difference image of the positive sample and the negative sample; Acquiring a human partial model by learning about the extracted feature set; And Calculating a difference image of the input image and using the human body model to detect a corresponding human portion of the difference image; Lt; / RTI > The step of obtaining the human- Learning the feature set of the difference image of the positive sample and the negative sample with respect to at least one pair of side faces of the plurality of side faces of the human body part to obtain the human body partial model And detecting the human body portion of the image. 12. The method of claim 11, Further comprising the step of determining the human part of the input image by removing errors based on the human geometry with respect to the human part after the step of detecting the human part. 12. The method of claim 11, Wherein the difference image is a difference image calculated on at least one scale in a horizontal direction, a vertical direction, a left-right diagonal direction and a right-left diagonal direction of the input image. 14. The method of claim 13, Wherein the extracting of the feature set extracts a feature set in at least one of the horizontal direction, the vertical direction, the left-right diagonal direction and the right-left diagonal direction using a single window or a plurality of windows. A partial detection method. 12. The method of claim 11, Wherein the human body part is a head part, a body part, a leg part, an arm part, or a part of a human body. 12. The method of claim 11, If the human body portion is a head, the positive sample and the negative sample of the head include positive and negative samples on the front, left half, left side, left rear half, rear half, right half and right half of the human head And detecting the human body part of the image. 17. The method of claim 16, The human body partial model includes: Learning about the feature set of the difference images of the positive and negative samples on the front, left half, left, rear half, rear, right half, right half and right half of the head based on the difference image of the head part as the human part A first head model obtained by advancing the first head model; A second head model obtained by learning a feature set of a difference image between a positive sample and a negative sample on the front and back sides of the head based on the difference image of the head; A third head model obtained by learning a feature set of a difference image between a positive sample and a negative sample on the left and right sides of the head based on the difference image of the head; And A fourth head model obtained by learning a feature set of a difference image between a positive sample and a negative sample on the left rear half side, the left half side, the right half side, and the right half side of the head part based on the difference image of the head part; And detecting a human body portion of the image. 18. The method of claim 17, Evaluating the detected head corresponding to the first head model to determine an error on the front and back sides of the head corresponding to the second to fourth head models, an error on the left and right sides of the head, Further comprising the step of removing errors in the left rear half, the left half, the right half, and the right half of the head. 12. The method of claim 11, Wherein in the step of detecting the human body part, detection is performed on the corresponding human body part in ascending order of the number of features of the feature set of a plurality of different human partial models. An exercise database storing positive and negative samples of the subject; An image processor for calculating a difference image for an input image; A sub-window processor for extracting a feature set from a difference image calculated by the image processor for positive samples and negative samples of an object stored in the practice database; An object classifier for detecting an object corresponding to the object model using the object model for the feature set; Lt; / RTI > Wherein the target classifier comprises: Characterized in that the object is detected using the object model obtained by learning the feature set of the difference image of the positive sample and the negative sample with respect to at least one of the plurality of side faces of the object, Object detection device. 21. The method of claim 20, Wherein the image processor calculates the input image as a difference image in a horizontal direction, a vertical direction, a left-right diagonal direction, and a right-left diagonal direction on at least one scale. 21. The method of claim 20, Wherein the sub window processor extracts the feature set from at least one of a difference image in a horizontal direction, a vertical direction, a left-right diagonal direction and a right-left diagonal direction using a single window or a plurality of windows Detection device. Storing a positive sample and a negative sample of the object and extracting a feature set from the difference image of the positive sample and the negative sample; Learning the extracted feature set to obtain a target model; And Calculating a difference image of the input image and using the target model to detect a corresponding one of the difference images; Lt; / RTI > The step of obtaining the target model comprises: Wherein learning is performed on a feature set of a difference image of a positive sample and a negative sample with respect to at least one or more sides of a plurality of sides of the object to obtain the object model And detecting a target of the image. 24. The method of claim 23, Wherein the difference image comprises a difference image calculated on at least one scale in the horizontal, vertical, left-right diagonal, and right-left diagonal directions of the image. 24. The method of claim 23, Extracting a set of features from at least one of a difference image in a horizontal direction, a vertical direction, a left-right diagonal direction and a right-left diagonal direction using a single window or a plurality of windows in extracting the feature set, . A photographing device for photographing an image of an object; A detecting device for detecting an object area corresponding to the object model from a difference image of the object using a target model obtained by learning on a feature set extracted from a difference image between the positive sample and the negative sample of the object; A state variable calculation device for calculating the state variable based on the object area in which the object exists among the images of the object detected by the detection device, thereby locating the object in the center area of the image; A control device for receiving the state variable from the state variable calculation device and adjusting the state; A storage device for storing an image of the object; And A display device for displaying an image of the object; The image forming system comprising: 27. The method of claim 26, Further comprising a marking device for providing a detection area of a target area manually marked by the user to the image of the object to the detection device. 27. The method of claim 26, Wherein the control device controls one of an operation of rotating, tilting, zooming and selecting a focus area according to the state variable. 27. The method of claim 26, Wherein the control device selects the new area having the object as a focus area. 27. The method of claim 26, Wherein the controller is configured to set a central area of the image as a default focus area, dynamically select a new area with the object as the focus area, and adjust the zoom, focal length, And controls the state variable of the tilt.

Images