JP5253195B2 - Object detection device - Google Patents

Description

  The present invention relates to an object detection apparatus that detects a detection target from an image.

  An object detection device that detects whether an image area corresponding to a detection target is included in an image is known.

  There has been proposed an object detection device that inputs an image to a discriminator that has previously learned whether or not the target object exists and detects the target object in the image. For example, Patent Document 1 discloses a technique for tracking a person using a particle filter, and using a cascade type discriminator and using the number of passing stages of the cascade as an indicator of humanity.

  Further, the identification device detects a face with a cascade-connected two-stage classifier. The first stage uses a classifier that does not require image normalization, and the second stage needs image normalization. A technique for improving processing speed by using a simple discriminator is disclosed.

JP 2008-26974 A JP 2005-108195 A

  By the way, since the conventional object detection apparatus learns the target object in advance, the learning result does not necessarily match the installation location of the object detection apparatus. For example, in a learning image, if a background image obtained by capturing an actual installation space of an object detection device is similar to an image determined to be human (such as installation of a mannequin or a hanger), it is erroneously detected as a person. There was a problem.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an object detection device that reduces false detections that occur as a result of prior learning that cannot be adapted to the installation location.

One aspect of the present invention includes an image acquisition unit that acquires the image, and a detection window region image in which a detection window is set from the image input from the image acquisition unit, and detects the detection window region image. A fixed discriminator that determines whether or not an object is present by calculating a degree of similarity indicating the degree of similarity between the detection window area image and the detection object; and the fixed identifier from a plurality of input images Whether the detection object exists based on the dictionary data that is updated by adding the similarity calculated from the detection window area of the new input image to the dictionary data that is a histogram of the similarity calculated by the classifier An object detection apparatus comprising: a learning discriminator that determines whether or not a detection target is present based on outputs from the fixed discriminator and the learning discriminator. It is.

  Here, it is preferable that the learning discriminator determines whether or not the detection target exists only for the detection window region image determined by the fixed classifier as having the detection target. is there.

The dictionary data is composed of first dictionary data learned from an image in which a detection target exists, and second dictionary data learned from an image in which no detection target exists, and the learning classifier Preferably, the second dictionary data is updated with the similarity output from the fixed discriminator when it is determined that the detection target does not exist.
The fixed classifier includes a cascade type classifier in which a plurality of strong classifiers that determine whether or not a detection target exists in the detection window region image are connected in series, and the learning classifier includes: It is preferable that the second dictionary data is updated with the similarity output from the strong classifier when it is determined that the detection target does not exist.

  ADVANTAGE OF THE INVENTION According to this invention, the false detection produced as a result of the prior learning which was not able to adapt to an installation place in an object detection apparatus can be reduced.

It is a functional block diagram which shows the structure of the object detection apparatus in embodiment of this invention. It is a figure explaining the detection process of an object. It is a figure which shows the structure of the discriminator in 1st Embodiment. It is a figure which shows the example of the similarity histogram in 1st Embodiment. It is a figure explaining the update method of the similarity histogram in 1st Embodiment. It is a flowchart which shows the process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the position setting process of the detection window area | region in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the identification process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the update process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the determination process in the object detection apparatus in 1st Embodiment. It is a figure which shows the structure of the discriminator in 2nd Embodiment. It is a figure which shows the example of the feature-value in 2nd Embodiment. It is a flowchart which shows the identification process in the object detection apparatus in 2nd Embodiment. It is a flowchart which shows the update process in the object detection apparatus in 2nd Embodiment. It is a flowchart which shows the initial setting process in the object detection apparatus in 2nd Embodiment.

<First Embodiment>
As shown in FIG. 1, the object detection apparatus 1 according to the embodiment of the present invention includes an image acquisition unit 2, a signal processing unit 3, a storage unit 4, and an output unit 5. The object detection device 1 acquires a monitoring image obtained by imaging a predetermined monitoring space, and detects a detection target such as a person or an object captured in the image. The image acquisition unit 2, the signal processing unit 3, the storage unit 4, and the output unit 5 are connected so as to be able to transmit information to each other.

  In this embodiment, an example in which a person shown in an image is a detection target will be described. However, the present invention is not limited to this, and the present invention can also be applied to the case of detecting articles such as commodities placed in distribution, vehicles passing through, and the like.

  The image acquisition unit 2 includes a so-called surveillance camera configured to include an imaging element such as a CCD element or a C-MOS element, an optical system component, an analog / digital converter, and the like. The image acquisition unit 2 may acquire an image via an Internet network. The image acquisition unit 2 transmits the captured image to the signal processing unit 3. The interval at which the images are acquired may not be a fixed time interval.

  The image is, for example, a color image having a width of 320 pixels, a height of 240 pixels, and each pixel expressing R (red), G (green), and B (blue) with 256 gradations.

  The signal processing unit 3 includes an arithmetic circuit such as a CPU, DSP, MCU, or IC. The signal processing unit 3 is connected to the image acquisition unit 2, the storage unit 4, and the output unit 5 so as to be able to transmit information. The signal processing unit 3 is a program that describes the processing in each means such as the detection window region selection means 30, the detection window control means 32, the fixed discriminator 34, the learning discriminator 36, the discriminator update means 37, and the judgment means 38. The computer is caused to function as each means by reading out from the storage unit 4 and executing it.

  As illustrated in FIG. 2, the signal processing unit 3 detects a detection target from the input image 200 from the image acquisition unit 2. For convenience of explanation, the upper left corner of the image 200 is the origin X = 0, Y = 0, the horizontal direction is the X axis, the vertical direction is the Y axis, the X axis is the right direction, and the Y axis is positively increased downward. And In the input image 200, a detection object (person) is shown at the positions of the areas 203 and 204, and rectangles 201 and 202 indicate detection window areas.

  The object detection apparatus 1 scans while gradually shifting the detection window area, and determines whether a person is captured in the detection window area. The arrow indicates the upper left coordinates when the detection window area is shifted, and the detection window area is scanned so as to search the entire image 200 without omission. The rectangles in the areas 203 and 204 indicate detection window areas (person candidate areas) that are determined to be human as a result of the detection process. In the vicinity of an image region where a person is shown, there may be a plurality of detection window regions that are determined to be humans. However, the final detection window region (in FIG. (Thick line: human area). However, the detection window area may be obtained by subjecting the reference background and the input image to difference processing, and labeling each image after the difference as the detection window area, or the entire image as the detection window area. A rectangle in the region 205 indicates a region where a hanger with clothes similar to a person who is a detection target is captured.

  The detection window area selection means 30 determines the width and height of the detection window area. The detection window area is scanned to determine whether or not a person is captured in the image while changing the width and height of the detection window area in order to correspond to people captured in various sizes in the image. Note that one or a plurality of widths and heights of the detection window region are stored in advance in the storage unit 4 in consideration of the size of the detection object on the image. However, even if the width and height are not stored in advance, they may be determined according to a predetermined rule.

  The detection window control means 32 determines an interval (scanning interval) for shifting the detection window region, and shifts the detection window region based on the determined scanning interval.

  The fixed discriminator 34 calculates the degree of similarity of how much the image in the detection window area resembles a person. The fixed discriminator 34 learns in advance using a large number of “person” image data and “non-person” image data.

  The similarity is calculated by the processing procedure shown in FIG. Specific examples of the fixed classifier 34, the learning classifier 36, and the classifier update means 37 shown in FIG. 1 are shown as a cascade type classifier 320, a learning classifier 330, and a classifier update means 332, respectively, in FIG.

  First, an image in the detection window region cut out from the input image is input to the cascade discriminator 320. The cascade type classifier 320 is a classifier in which a plurality of strong classifiers such as the strong classifiers 321, 322, and 323 are arranged in series (here, an example in which N strong classifiers are arranged in series is shown). ). Individual strong classifiers are Histograms of Oriented Gradients (HOG) features (Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, In Proceedings of IEEE Conference Computer Vision and Pattern Recognition 2005) To learn in advance with AdaBoost. That is, various images of people who are detection objects and images that do not show people are prepared in large quantities, each image is correctly identified as to whether it is an image of the detection object, and these data To learn with Adaboost so that both can be identified. When calculating the similarity, each strong classifier calculates the HOG feature from the input image, and calculates the similarity from the feature amount selected by Adaboost.

  The order in which the strong classifiers 321 to 323 are calculated in advance is determined, and the strong classifier 321 to be calculated first is 1, the next strong classifier 322 to be calculated is 2. Assign the same number as the order of calculation.

  Each of the strong discriminators 321 to 323 receives the image cut out as the detection window region and calculates the similarity. The strong classifiers 322, 323, etc. other than the first strong classifier 321 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). When the value is equal to or less than the threshold value (arrow F in the figure), the classifier update unit 332 updates the learning classifier 330. The threshold is set to 0, for example, and if it is greater than 0, it is similar to a person, and if it is 0 or less, it is determined that it is not similar to a person. Hereinafter, the number of the strong classifier for which the similarity is calculated is referred to as the number of determination steps.

  The learning discriminator 330 uses the similarity of each strong discriminator of the cascade discriminator 320 as an input to determine whether or not the person is a person.

  The learning discriminator 330 performs the determination only when the cascade type discriminator 320 determines that the last strong discriminator 323 is a person (arrow T in the figure). Only when the possibility of “person” is high in the cascade type classifier 320, that is, when the similarity obtained by the strong classifier 323 at the final stage is larger than the threshold (arrow T in the figure), the learning classifier 330 makes the determination. I do. The learning discriminator 330 receives the similarity Sn (n is 1 to M: M is the number of determination stages (1 ≦ M ≦ N)) obtained by the strong classifiers up to the number of determination stages in the cascade classifier 320 as an input. A similarity vector S = (similarity of the first strong classifier, similarity of the second strong classifier,..., A vector representation of the similarities obtained by the strong classifiers of the cascade type classifier 320 arranged side by side. (Similarity of Mth strong classifier) = (S1, S2,..., SM).

  Calculating the probability P (person | S) of “person” when the similarity vector S is obtained, and the probability P (non-person | S) of “non-person” when the similarity S is obtained; If P (person | S)> P (non-person | S), it is determined as “person”, otherwise it is determined as “non-person”.

Calculations of P (person | S) and P (non-person | S) can be obtained by the formulas (1) and (2) from Bayes' theorem.

  Here, P (S | People) obtains the probability (likelihood) of obtaining the similarity S in the case of “People”, and P (S | People) obtains the similarity S in the case of “Non-People”. Probability (likelihood). P (person) is a prior probability that indicates the probability of “person” being included in the detection window area, and similarly P (other than person) is included in the detection window area with a probability of “non-person”. It is a prior probability that represents whether or not. Since the probability that “person” and “non-person” are included in the detection window region in advance is not known, it is assumed that the prior probability P (person) = P (non-person) = 0.5. Since P (non-person | S) = 1−P (person | S), if P (person | S) is obtained, P (non-person | S) is easily obtained. Hereinafter, P (person | S) is referred to as the similarity in the learning discriminator 330.

When a random variable vector s having the similarity of each strong classifier as an element is prepared, the distribution P (s | person) of the similarity vector s of “person” and the distribution P (s) of the similarity vector s of “non-person”. If (other than people) is obtained, P (people | s) can be calculated. Although the similarity vector s is multi-dimensional data, assuming that the similarity sn of each strong classifier is independent, P (s | people) and P (s | others) are expressed by equations (3) and (4). Desired.

  Here, P (sm | person) is the distribution of similarity of “m” of the mth strong classifier, and P (sm | other) is the similarity of “mother” of the mth strong classifier. Distribution.

  In the present embodiment, it is expressed by a histogram such that the sum of similarity distributions of all bins (separators) is 1. Each histogram is stored in the similarity histogram 42 of the storage unit 4. However, the similarity distribution is not limited to the histogram, and may be expressed by a normal distribution, a mixed normal distribution, or the like.

  The “person” similarity histogram 42 is configured by obtaining a similarity distribution of each strong classifier using learning data that is an image including “person” prepared in advance. In the present embodiment, the “person” similarity histogram 42 is not sequentially learned but may be updated. The similarity histogram 42 of “non-person” assumes that no person is included in the first few frames when the surveillance camera is installed, and the similarity distribution of the strong classifiers obtained at that time is shown. Configure by asking. Thereafter, when the detection window region is determined to be “non-human” in the cascade type discriminator 320 or the learning discriminator 330, the similarity histogram 42 of “non-human” is used by using the similarity obtained by the cascade type discriminator 320. Are updated sequentially.

  FIG. 4 shows an example of the similarity histogram 42 when the number of strong classifiers in the cascade classifier 320 is N = 2. The similarity histogram 401 is a distribution of similarity “person” 402: P (s1 | person) and similarity distribution 403: P (s1 | person) of “non-person” in the similarity s1 of the first strong classifier. Except). The similarity histogram 411 shows the distribution of similarity of “person” 412: P (s2 | person) and the distribution of similarity of “non-person” 413: P (s2 | person) in the similarity s2 of the second strong classifier Except). The four similarity histograms 402, 403, 412, and 413 are normalized so that the sum of the similarity distributions of all bins is 1.

  When the determination is made only with the cascade classifier 320, if the degree of similarity obtained by each strong classifier is larger than a predetermined threshold (for example, set to 0), it is determined as “person”. Then, the probability is calculated from the obtained similarity of each strong classifier and the past similarity histogram 42 and can be adaptively determined.

  As an example, when the similarity distribution of “person” and “non-person” is represented by the similarity histograms 401 and 411 shown in FIG. 4, the similarity of the first strong classifier is S1 = 1.0, A case where the similarity of the second-stage strong classifier is S2 = 1.0 will be described. When the threshold values in the first and second stage strong classifiers are 0, the similarity exceeds the threshold value in both the first and second stage strong classifiers. It is determined as “person”. On the other hand, in the learning discriminator 330, the probability of “person” when the similarity vector S = (1.0, 1.0) is obtained: P (person | S = (1.0, 1.0) ) = P (S1 = 1.0 | People) × P (S2 = 1.0 | People) × P (People) / A and the probability of “Non-People”: P (Non-People | S = (1. 0, 1.0)) = 1−P (person | S = (1.0, 1.0)). However, A = P (S | person) × P (person) + P (S | other than person) × P (other than person). Based on the histograms 401 and 411 shown in FIG. 4, P (person | S = (1.0,1.0)) <P (non-person | S = (1.0,1.0)). The discriminator 330 determines “other than human”.

  If a similarity histogram matching the characteristics of the installation environment of the object detection device can be constructed, the learning discriminator 330 can make a more accurate determination based on the similarity calculation result from the cascade discriminator 320.

  After the determination by the learning discriminator 330, the learning discriminator 330 is updated by the discriminator update unit 332. Further, the similarity management unit 334 associates the similarity, the size (width and height) of the detection window region, and the center coordinates, and stores them in the storage unit 4 as the determination information history 40.

  The discriminator update unit 332 updates the similarity histogram of the learning discriminator 330. In the present embodiment, the learning classifier 330 is updated by the classifier updating means 332 when the degree of similarity obtained by each strong classifier is equal to or less than a threshold value (arrow F in FIG. 3). In addition, even if the similarity determined by the strong classifier exceeds the threshold value in the cascade classifier 320 (the arrow T in FIG. 3), the learning classifier 330 determines “other than human” (FIG. 3). In the case of the middle F arrow), the classifier update unit 332 updates the learning classifier 330. That is, only the “non-person” similarity histogram 42 is updated, and the “person” similarity histogram 42 is not updated.

  The discriminator update unit 332 updates the similarity histogram 42 obtained up to one time ago with the data obtained at the current time. For example, as shown in FIG. 5, α blending is used for the update process by the discriminator update unit 332. FIG. 5 shows a process of updating the n-th “non-human” similarity histogram 42 of the learning discriminator 330, based on the similarity histogram 501 before the update up to the previous time and the data obtained at the current time. An updated similarity histogram 502 is shown.

  When it is determined whether or not an image of a person is included in the detection window area, it is determined that the cascade discriminator 320 determines “person” and the learning discriminator 330 determines “other than human”. A method of updating the similarity histogram 501 corresponding to the nth strong classifier when the similarity of the nth strong classifier is 0.8 is shown. First, each bin of the similarity histogram 501 before update is multiplied by (1.0−α). Thereafter, a weight corresponding to α is added to the bin corresponding to the similarity of 0.8 obtained at the current time (the hatched portion of the bin 503 in the figure). α is a predetermined parameter. When α is large, the influence of past information is reduced earlier, and when α is small, the influence of past information is longer. When updating for the first time, that is, when the similarity histogram 501 is empty, the bin value corresponding to the obtained data is set to 1. However, since the influence of the first obtained data is large in this way, α may be set to a relatively large value while the number of updates is small, and may be set to a smaller value as the number of updates increases. Good.

  Here, referring back to FIG. 1, the determination means 38 uses the size (width, height) and center coordinates of the detection window region stored as the determination information history 40, and the similarity of the learning discriminator 330. Finally, determine where the person is in the image. In the data included in the determination information history 40, a detection window region whose similarity of the learning discriminator 330 is equal to or higher than a threshold (for example, set to 0.8) is extracted as a human candidate region. If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. The detection window region having the highest similarity of the learning discriminator 330 is selected as the human region among the grouped human candidate regions. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the similarity management unit 334, and stores, as the determination information history 40, three pieces of information including the size (width and height) of the detection window region, the center coordinates, and the similarity of the learning discriminator 330 in association with each other. To do. For each strong classifier included in the cascade classifier 320, a similarity histogram 42 of “person” and “non-person” is stored in association with the number of each strong classifier. The similarity histogram 42 corresponds to the dictionary data, the similarity histogram of “person” is the first dictionary data learned from the image in which the detection target exists, and the similarity histogram of “non-person” is the detection target. This corresponds to the second dictionary data learned from the non-existent image.

  The output unit 5 can be configured to include a sound output unit that outputs a notification sound and a display unit that displays an input image. When the determination unit 38 detects a human area, an alarm is sounded by a sound output unit such as a speaker or a buzzer, or an input image is displayed on an external display device such as a display. The output unit 5 may include an interface for connecting the computer to a network or a telephone line. In this case, the output unit 5 sends an input image and human area information to a center device (not shown) via information transmission means such as a telephone line or the Internet. The center device is a host computer installed in a center or the like that monitors a detection target in an image.

  Hereinafter, the processing in the object detection apparatus 1 will be described with reference to the functional block diagram of FIG. 1 and the flowchart of FIG.

  In step S <b> 10, an image is acquired by the image acquisition unit 2, and the image is transmitted to the signal processing unit 3. The image acquisition timing is a fixed time interval.

  In step S20, the size (width and height) of the detection window region is determined. The entire image is scanned with each size of the detection window region while sequentially changing the size of the detection window region to a plurality of preset sizes. In this embodiment, since the detection window region is rectangular, it is only necessary to determine the width and height as the size. However, the detection window region may have an arbitrary shape, and in that case, the shape and size are determined. To decide. This processing is performed by the detection window region selecting means 30.

  The processing from step S30 to step S50 is repeated until the entire image has been scanned with the size of the detection window region set in step S20.

  In step S30, the position of the detection window area is determined. This process is performed by the detection window control means 32. This process is performed according to the flowchart shown in FIG. The upper left coordinate of the detection window area is called a start point, the start point of the detection window area to be determined is (SX, SY), and the previous start point of the detection window area is (BX, BY).

  In step S301, it is determined whether or not the size of the detection window area has been changed. If the size of the detection window area has just been changed, the start point (SX, SY) of the detection window area is set to (0, 0) in step S305, and the process proceeds to step S40. If it is not the first process after the size of the detection window area is changed, the process proceeds to step S302.

  In step S302, the start point (BX, BY) of the previous detection window area is read. In step S303, it is determined whether scanning has been performed from the start point (BX, BY) of the previous detection window area read in step S302 to the right end of the image. When scanning to the right edge of the image is completed in the detection window area, that is, when the right edge BX + W (W is the width of the detection window area) of the detection window area coincides with the right edge of the image, the detection window area in step S306 Is set to (SX, SY) = (0, BY + q), and the process proceeds to step S40. However, q is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, q = 4 pixels is set. If not scanned to the right end, the process proceeds to step S304.

  In step S304, the detection window area is shifted by p pixels in the X direction, and (SX, SY) is determined as the start point of the detection window area so as not to be shifted in the Y direction. That is, the detection window region start point (SX, SY) = (BX + p, BY). However, p is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, p = 4 pixels is set. Thereafter, the process proceeds to step S40.

  In step S40, a similarity indicating how much the image in the detection window region set in step S30 resembles a person is obtained. This is the discrimination processing of the cascade type discriminator 320 and the learning discriminator 330 shown in FIG. The identification process will be described with reference to the flowchart of FIG.

  In step S401, the HOG feature amount is calculated from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  In step S402, it is determined whether or not all strong classifiers have been investigated. If all the strong classifiers have been investigated, the process proceeds to step S406. If not, the process proceeds to step S403, and the next strong classifier is investigated. In step S403, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S404, the similarity calculated in step S403 is stored in the temporary storage area together with the number of the strong classifier that has been determined. This is used in similarity calculation in the learning discriminator 330 in step S407 and update processing in step S50.

  In step S405, it is determined whether the similarity calculated in step S403 is greater than a preset threshold value. If it is larger than the threshold value, the process proceeds to step S402. If it is equal to or smaller than the threshold value, the similarity of a learning discriminator 330 described later is stored as a value equal to or smaller than the threshold value in step S501, and the process proceeds to step S50.

  Steps S406 to S408 are processing of the learning discriminator 330. In step S406, among the similarity histograms 42 of the strong classifiers up to the number of determination stages of the cascade classifier 320, the similarity histogram 42 of “non-human” whose update count is equal to or greater than the reference value is extracted. This is because if the similarity histogram 42 with a small number of updates is used, the reliability of the similarity calculated by the learning discriminator 330 becomes low. Therefore, the similarity in the learning discriminator 330 is calculated using the similarity histogram 42 updated more than the reference value and the corresponding similarity of the strong discriminator stored in step S404.

  In step S407, the learning discriminator 330 calculates the similarity. The similarity is calculated using P (person | S) in the formula (1). In step S408, the similarity calculated in step S407 is stored in the storage unit 4 as the determination information history 40 in association with the size (width and height) of the detection window region and the center coordinates, and the process proceeds to step S50.

  In step S50, similarity histogram update processing is performed. This is done by the discriminator update means 37. The update process is performed according to the flowchart of FIG. In the present embodiment, the similarity histogram 42 of “non-person” is updated, and the similarity histogram 42 of “person” is not updated.

  In step S501, if the similarity calculated by the learning discriminator 330 in step S407 is greater than the threshold value, it is highly likely that the person is “person”, so the similarity histogram 42 is not updated, and the process proceeds to step S60. .

  In step S502, the “non-human” similarity histogram 42 corresponding to each strong classifier included in the cascade classifier 320 is updated with the similarity stored in step S404 of FIG. Accordingly, only the “non-human” similarity histogram 42 corresponding to the strong classifier used for the current determination is updated. The update method is as described for the discriminator update means 332. After the update process, the process proceeds to step S60.

  In step S60, it is determined whether or not the detection process has been completed for detection window regions of all sizes. If the inspection is completed for all detection window regions, the process proceeds to step S70, and if not, the process returns to step S20.

  In step S70, a person is finally captured from the size (width and height) of each detection window region stored in the determination information history 40 obtained in step S50, the center coordinates, and the similarity of the learning discriminator 330. Determine the position. This processing is performed by the determination means 38. The determination process is performed according to the flowchart of FIG.

  The processing of steps S701 to S702 is performed for all detection window regions stored as the determination information history 40 obtained in step S50. In step S701, it is determined whether or not the similarity of the learning discriminator 330 obtained by the learning discriminator 330 is greater than a threshold value. If the condition is satisfied, the process proceeds to step S702. Otherwise, the next detection is performed. Judge the window area. In step S702, the detection window area that satisfies the condition in step S701 is added to the human candidate area.

  As shown in FIG. 2, candidate candidate areas obtained by repeating steps S701 to S702 on the information stored in the determination information history 40 are areas 203 and 204 near the area where the person is shown. May be extracted multiple times. In steps S703 to S705, a human region is finally selected from a plurality of human candidate regions.

  In step S703, using the size and center coordinates of the detection window area extracted as the human candidate area, the human candidate areas that overlap a certain area (for example, more than half the area of the detection window area) are grouped together. . In step S704, the detection window region having the highest similarity of the learning discriminator 330 is selected as a human region among the human candidate regions for each group created in step S703. In step S705, the detection window area selected in step S704 is set as a human area, and the process proceeds to step S80.

  In step S80, if there is at least one human area based on the human area determined in step S70, the detection window information of the determined human area is output as an abnormal signal together with the image. This processing is performed by the output unit 5.

[Initial settings]
The initial setting process when the surveillance camera is installed will be described below. At the time of initial setting, the similarity histogram 42 of “non-human” corresponding to each strong classifier is registered under the assumption that there is no person in the imaging area of the surveillance camera. It is assumed that the initial setting process is completed after a predetermined number of images are processed after the monitoring camera is installed.

  The initial setting process is a process in which steps S40 and S50 are slightly changed except for steps S70 and S80 in the flowchart of the detection process of FIG. Further, only differences between the processes of steps S40 and S50 will be described.

  In step S40, the processing from step S401 to S405 in FIG. 8 is performed. If the investigation has been completed for all strong classifiers in step S402 (YES in step S402), the processing in steps S406 to S408 is not performed. The process proceeds to S50. That is, the difference is that the processes of steps S406 to S408 are omitted in the initial setting process.

  In step S50, since it is assumed that no person is in the monitoring space in the initial setting process, the determination process in step S501 is not performed. In step S502, the similarity histogram 42 of “non-human” of the corresponding strong classifier is updated using the similarity of the strong classifier up to the number of determination stages obtained in step S40, and then the process proceeds to step S60. .

  In addition, the similarity histogram 42 of “non-human” is not sequentially updated every time an image is acquired, but the similarity calculated by the cascade discriminator 320 for a predetermined number of images is stored, and similarities are collectively obtained. The degree histogram 42 may be created.

<Second Embodiment>
The object detection apparatus according to the second embodiment performs the determination of the fixed classifier and the learning classifier in parallel, and integrates the respective determination results. In the second embodiment, processing in the fixed discriminator 34, the learning discriminator 36, the discriminator update unit 37, and the determination unit 38 in the first embodiment is different and will be described. In the storage unit 4, the similarity histogram 42 is stored in the first embodiment, but in the second embodiment, it will be described as the learning discriminator feature amount distribution 42.

  The fixed discriminator 34 obtains a similarity indicating how much the image in the detection window area resembles a person. This is the same processing as in the first embodiment. Specific examples of the fixed classifier 34, the learning classifier 36, and the classifier update means 37 shown in FIG. 1 are shown as a cascade type classifier 620, a learning classifier 632, and a classifier update means 636, respectively.

  The cascade type discriminator 620 in FIG. 11 includes a plurality of strong discriminators 621 to 623, and uses an image cut out as a detection window region as an input to calculate the similarity. Strong discriminators 622, 623, etc. other than the strongest discriminator 621 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). For example, the threshold is set to 0, and if the similarity obtained by the strong classifiers 621 to 623 is greater than 0, it is similar to a person, and if it is 0 or less, it is determined not to be similar to a person. When the similarity obtained by the strong classifier 323 at the final stage is larger than the threshold (arrow T in the figure), or when the similarity obtained by each strong classifier is less than or equal to the threshold (arrow F in the figure). In addition, the determination stage number and similarity of the cascade discriminator 620 are output to the determination information history unit 630. The determination information history unit 630 associates the output determination step number and similarity with the size and center coordinates of the detection window region and stores the determination information history 40 in the storage unit 4.

  Unlike the first embodiment, the learning discriminator 632 receives an image and determines whether it is a person. The learning discriminator 632 extracts a feature amount related to the edge from the input image of the detection window region. First, an edge image is created from the image of the detection window area. The edge image may be obtained for the entire monitoring image in advance. For example, as shown in FIG. 12, the detection window area is divided into a total of eight rectangular areas divided into two horizontal parts and four vertical parts, and a histogram relating to the edge direction is created for each area. For example, the edge direction is set to nine directions, and a histogram 702 in which the edge strength is weighted for each edge direction in each region is created. Note that since the edges that are inverted by 180 degrees are regarded as the same direction, the directions that the histogram 702 can take are 0 to 180 degrees, and when assigned to 9 directions, the increments are 20 degrees.

  Thus, the histogram 702 is created in each region. The bins of the histogram 702 form a 72-dimensional vector with 9 directions × 8 regions. Finally, normalization is performed so that the norm of the 72-dimensional vector is 1. The vector created in this way is used as the feature amount X of the detection window region.

  The learning discriminator 632 determines whether the detection window area includes “person” or “other than person” based on the feature amount X. In order to determine whether or not a person is a person from the obtained feature quantity X, the probability P (person | X) that is “person” when the feature quantity X is obtained, and the “person” when the feature quantity X is obtained. The probability P (non-human | X) that is “non-human” is calculated, and if P (person | X)> P (non-human | X), it is determined as “human”, otherwise it is determined as “non-human”.

The calculation of P (person | X) and P (non-person | X) can be obtained by the formulas (5) and (6) from Bayes' theorem.

  Here, P (X | person) is the probability (likelihood) that the feature quantity X is obtained for “person”, and P (X | other) is the feature quantity X for “non-person”. Probability (likelihood). P (person) is a prior probability that indicates the probability of “person” being included in the detection window area, and similarly, P (other than person) is the probability of “non-person” being included in the detection window area. Is a prior probability. Since the probability that “person” is included in the detection window region is not known in advance, it is assumed that the prior probability P (person) = P (other than person) = 0.5. Since P (non-person | X) = 1−P (person | X), if P (person | X) is obtained, P (non-person | X) can be easily obtained. Hereinafter, P (person | X) is referred to as the similarity in the learning discriminator 632.

  In the present embodiment, the distribution of P (X | people) and P (X | people) is expressed as a mixed normal distribution. The feature quantity distribution P (X | person) of “person” calculates the feature quantity from prepared learning data of “person”, and uses the EM (Expectation-Maximization) algorithm (AP Dempster, NM) Laird, and DB Rubin. “Maximum likelihood from incomplete data via the EM algorithm”, Journal of the Royal Statistical Society, Series B, 39 (1), 1-38, 1977). The feature quantity distribution P (X | non-person) of “non-human” calculates the feature quantity from a plurality of images taken on the assumption that there is no “person” at the time of installation of the surveillance camera, and the mixed normal distribution by the EM algorithm Determine the parameters.

  After the similarity is obtained by the learning discriminator 632 in FIG. 11, the size (width, height) of the detection window region and the center coordinates and the similarity obtained by the learning discriminator 632 are obtained by the learning similarity management unit 634. The determination information history 40 is associated and stored in the storage unit 4.

  Next, the discriminator update unit 636 updates the learning discriminator 632 based on the similarity of the cascade discriminator 620, the number of determination stages, and the result of the similarity of the learning discriminator 632. In the present embodiment, the discriminator update unit 636 only selects “person” based on the number of determination stages and the similarity of the cascade discriminator 620 and the similarity of the learning discriminator 632. "Non-other" feature quantity distribution P (X | other than people) is updated. However, the feature amount distribution P (X | person) of “person” may be updated only when the possibility of “person” is high.

  The “non-human” feature quantity distribution P (X | non-human) is modeled as a mixed normal distribution, and the feature quantity distribution P (X | human) is obtained each time one piece of data determined as “non-human” is obtained. Radford Neal, RM and GE Hinton (1998) "A new view of the EM algorithm that justifies incremental, sparse and other variants", Learning in Graphical Models, pp.355-368. Kluwer Academic Publishers ), The E step and the M step are executed once to update.

  The determination unit 38 determines the size (width, height) and center coordinates of the detection window region stored as the determination information history 40, the number of determination stages and the similarity of the cascade type classifier 620, and the similarity of the learning classifier 632. To finally determine where the person is in the image.

  Among the data included in the determination information history 40, the number of determination stages of the cascade type discriminator 620 is the final stage, the similarity is larger than a threshold (for example, set to 0), and the similarity of the learning discriminator 632 is a threshold ( For example, detection window regions that satisfy all of the above conditions are set as human candidate regions. If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. The detection window region having the highest similarity of the cascade discriminator 620 is selected as the human region among the grouped human candidate regions. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5. The selection as a human region may use the similarity of the learning discriminator 632 without using the similarity of the cascade discriminator 620.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the learning similarity management unit 634 and stores the determination information history 40 and the learning discriminator feature amount distribution 42. The determination information history 40 stores information related to the size (width, height) and center coordinates of the detection window region, the number of determination stages and the similarity of the cascade discriminator 620, and the similarity of the learning discriminator 632 in association with each other. It is. The learning identifier feature quantity distribution 42 is a human feature quantity distribution P (X | person) and a non-person feature quantity distribution P (X | other person). Specifically, parameters related to the mixed normal distribution of “people” and “non-people”, that is, the mixture ratio, average value, and covariance matrix of each normal distribution are stored. The learning discriminator feature quantity distribution 42 corresponds to the dictionary data, and the human feature quantity distribution P (X | person) is the first dictionary data learned from the image in which the detection target exists, the non-human feature quantity distribution. P (X | other than people) respectively corresponds to the second dictionary data learned from the image in which the detection target does not exist.

  Hereinafter, the process in this Embodiment is demonstrated. Since the processing in the present embodiment is almost the same as the processing shown in the flowchart of FIG. 6, only different steps S40, S50, and S70 will be described.

  In step S40, a similarity indicating how much the image in the detection window area set in step S30 resembles a person is obtained. This is processing in the cascade discriminator 620 and learning discriminator 632 shown in FIG. 11, and will be described with reference to the flowchart of FIG. Steps S451 to S455 are processing in the cascade discriminator 620, and steps S456 to S458 are processing in the learning discriminator 632.

  In step S451, the HOG feature amount is extracted from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  In step S452, it is determined whether or not all strong classifiers have been investigated. If all strong classifiers have been investigated, the process proceeds to step S455, and if not complete, the process proceeds to step S453, and the next strong classifier is investigated. In step S453, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S454, it is determined whether the similarity calculated in step S453 is greater than a preset threshold (for example, set to 0). If it is greater than the threshold value, the process proceeds to step S452, and if it is less than the threshold value, the process proceeds to step S455.

  In step S455, the similarity calculated last in step S453 and the number of determination steps thereof are stored in the storage unit 4 as the determination information history 40 in association with the size (width, height) of the detection window region and the center coordinates. Here, the similarity calculated last becomes the similarity of the fixed classifier. In step S458, the similarity of the learning discriminator 632 is also stored.

  In step S456, the feature quantity X used in the learning discriminator 632 is obtained. This is as described above. In step S457, the similarity of the learning discriminator 632 is calculated based on the feature amount X obtained in step S456. The degree of similarity P (person | X) can be obtained from Equation (5). In step S458, the similarity of the learning classifier 632 calculated in step S457 is stored in the storage unit 4 as the determination information history 40 in association with the similarity of the fixed classifier stored in step S455. Thereafter, the process proceeds to step S50.

  In step S50, the learning discriminator feature quantity distribution is updated. This is performed by the discriminator update means 636. The update process is performed according to the flowchart shown in FIG. In the present embodiment, the feature quantity distribution of “non-person” is updated, and the feature quantity distribution of “person” is not updated.

  In step S551, it is determined whether or not it is “other than a person”. “Non-human” means that the number of determination stages of the cascade discriminator 620 is equal to or smaller than the threshold T1 (1 ≦ T1 ≦ N), the similarity of the cascade discriminator 620 is smaller than the threshold T2, and the similarity of the learning discriminator 632 Indicates a case where all the conditions that the value is smaller than the threshold value T3 are satisfied, that is, a case where it is surely “non-human”. Only when it is determined as “non-human”, the learning discriminator feature distribution 42 is updated. If the condition is satisfied, the process proceeds to step S552, and if not, the process proceeds to step S60.

  In step S552, the feature quantity distribution 42 of “non-human” is updated. Using the feature amount X obtained from the detection window area at the current time, the mixed normal distribution P (X | other than people) other than “people” is updated using the sequential EM algorithm. Thereafter, the process proceeds to step S60.

  In step S70, an area in which a person is finally captured is determined based on the determination information history 40 obtained in step S50. This processing is performed by the determination means 38. The process in step S70 differs from the process in the first embodiment along the flowchart shown in FIG. 10 only in the condition in step S701. In step S701, only when the number of determination stages of the cascade discriminator 620 is the final stage, the similarity of the cascade discriminator 620 is greater than the threshold, and all the conditions of the similarity of the learning discriminator 632 are greater than the threshold. In S702, a human candidate area is set.

[Initial settings]
The initial setting process when the surveillance camera is installed in the present embodiment will be described below. At the time of initial setting, on the assumption that there is no person in the imaging area of the surveillance camera, the feature distribution for learning discriminator 42: P (X | other than person) of “non-person” is created. The feature quantity distribution P (X | person) for the learning discriminator “person” is created by obtaining the mixture ratio, average, and covariance matrix of each normal distribution of the mixed normal distribution from previously collected learning data.

  In the initial setting process, a feature quantity X is obtained from a predetermined number of images after the monitoring camera is installed, and the feature quantity distribution P for learning discriminator P (X | non-person) is determined from these feature quantities X. And finish.

  The initial setting process is performed according to the flowchart of FIG. Steps S101 to S103 are the same as steps S10 to S30. In step S <b> 104, the learning discriminator feature amount X is created and stored in the temporary storage area of the storage unit 4. Step S105 is the same as step S60.

  In step S106, it is determined whether or not a predetermined number of images have been processed. If a predetermined number of images have been processed, the process proceeds to step S107. Otherwise, the process returns to step S102.

  In step S107, the feature amount X accumulated in step S104 is used to estimate the feature amount distribution 42: P (X | non-person) of “non-person”. The feature amount distribution P (X | other than human) is represented by a mixed normal distribution composed of a sum of a predetermined number of normal distributions, and the mixture ratio, average, and covariance matrix of each normal distribution are obtained by the EM algorithm. .

  Moreover, although the said embodiment demonstrated the aspect which implement | achieves the function of each part of the object detection apparatus 1 with one computer, it is not limited to this. The functions of each part of the object detection device 1 can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by a single computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Image acquisition part, 3 Signal processing part, 4 Memory | storage part, 5 Output part, 30 Detection window area | region selection means, 32 Detection window control means, 34 Fixed discriminator, 36 Learning discriminator, 37 Discriminator update Means, 38 judgment means, 40 judgment information history, 42 learning feature distribution (similarity histogram), 200 input image, 201, 202 rectangle, 203, 204 area, 310 detection window cutting means, 320 cascade type discriminator , 321, 322, 323 strong classifier, 330 learning classifier, 332 classifier update unit, 334 similarity management unit, 401,411 similarity histogram, 402,403 distribution, 411 similarity histogram, 412, 413 distribution, 501 , 502, 503 Histogram, 620 Fixed classifier, 621, 622, 623 Strong classifier, 6 30 judgment information history means, 632 learning discriminator, 634 learning similarity management means, 636 discriminator update means, 702 histogram.

Claims (4)

An object detection device for detecting a detection target from an image obtained by imaging a monitoring area,
An image acquisition unit for acquiring the image;
A detection window area image in which a detection window is set is captured from the image input from the image acquisition unit, and whether or not a detection target exists in the detection window area image is determined based on the detection window area image and the detection window area image. A fixed classifier that calculates and determines the degree of similarity indicating the degree of similarity of the detection object ;
The dictionary data, which is a histogram of the similarity calculated by the fixed classifier from a plurality of input images, is updated by adding the similarity calculated from the detection window area of the new input image, and is detected based on the dictionary data. A learning discriminator for determining whether an object exists;
Determining means for determining whether a detection target exists based on outputs from the fixed classifier and the learning classifier;
An object detection apparatus comprising:   The learning discriminator determines whether or not a detection target exists only for a detection window region image determined by the fixed classifier as having a detection target. The object detection apparatus according to 1. The dictionary data is composed of first dictionary data learned from an image in which a detection target exists and second dictionary data learned from an image in which no detection target exists,
The said learning discriminator updates the said 2nd dictionary data by the similarity output from the said fixed discriminator when it determines with a detection target object not existing, The Claim 1 or 2 characterized by the above-mentioned. The object detection apparatus described. The fixed discriminator has a cascade type discriminator in which strong discriminators that determine whether or not a detection target exists in the detection window region image are connected in a plurality of stages in series.
The said learning discriminator updates the said 2nd dictionary data by the similarity output from the said strong discriminator when it determines with a detection target object not existing. Object detection device.

Images