JP5253194B2 - Object detection device - Google Patents

Description

  The present invention relates to an object detection apparatus that detects a detection target from an image.

  An object detection device that detects whether an image area corresponding to a detection target is included in an image is known.

  Conventionally, there has been proposed an object detection apparatus that learns an abnormal image from an input image in real time and detects an abnormality in an image obtained by photographing a monitoring target. For example, Patent Document 1 discloses a detection device that performs sequential learning on continuously input images and detects unsteady images in real time, and uses unsupervised dictionary data to determine the degree of unsteadiness. A detection device that learns and updates at any time using a learning method is disclosed.

JP 2007-299250 A

  In the prior art, since the entire image is used as a learning unit of the object detection device, it is not possible to perform learning suitable for each partial region of the image. For example, when learning is performed from one image having a plurality of partial areas including images of different objects, such as an image having an area where a hung image is reflected and an area where only a wall is reflected. Also, learning is performed in units of the entire image without distinction for each partial region. When such a learning result is used, there is a limit in improving the accuracy of detection of the detection target in the image.

  An object of the present invention is to provide an object detection apparatus using a discriminator that can be updated in accordance with an image area.

  An object detection apparatus according to an aspect of the present invention is an object detection apparatus that detects a detection target from an image obtained by capturing a monitoring region, and the image acquisition unit that acquires the image and the input from the image acquisition unit A detection window area selecting means for taking in a detection window area image from detection windows set sequentially in the image; a discriminator setting means for setting a discriminator corresponding to a setting position of the detection window; and a detection target in the detection window area image The discriminator for determining whether or not an object exists, and the discriminator update means for updating the discriminator set by the discriminator setting means based on information calculated in the detection window region image, It is characterized by having.

  The discriminator includes a cascade discriminator in which strong discriminators that determine whether or not a detection target exists in the detection window region image are connected in series, and the discriminator update unit includes the strong discriminator. If there is a stage that does not use, the number of stages used by the cascade discriminator may be reduced.

  The discriminator includes a learning discriminator that determines whether or not a detection target exists in the detection window region image using information learned according to a set position of the detection window from an image obtained by photographing the monitoring region. The discriminator update means increases the number of stages of use of the cascade discriminator when the determination result of the cascade discriminator and the determination result of the learning discriminator do not match. You may let them.

  The discriminator is a learning discriminator that determines whether or not a detection target exists in the detection window area image using information learned according to a set position of the detection window from an image obtained by photographing the monitoring area. The discriminator update means may perform learning of the learning discriminator using the detection window region image.

  Further, the setting position of the detection window that is sequentially set corresponds to one of the divided areas obtained by dividing the image into a plurality of areas in advance, and the classifier setting unit corresponds to the setting position of the detection window. It is preferable to set the same classifier for each of the divided areas.

  ADVANTAGE OF THE INVENTION According to this invention, the object detection apparatus using the discriminator which enabled the update according to the area | region of an image can be provided.

It is a functional block diagram which shows the structural example of the object detection apparatus in embodiment of this invention. It is a figure explaining the detection process of an object. It is a figure which shows the structural example of a discriminator. It is a figure which shows the example of a similarity histogram. It is a figure explaining the update method of a similarity histogram. It is a flowchart which shows the example of the process in an object detection apparatus. It is a flowchart which shows the example of the position setting process of a detection window area | region in an object detection apparatus. It is a flowchart which shows the example of the identification process in an object detection apparatus. It is a flowchart which shows the example of the update process in an object detection apparatus. It is a flowchart which shows the example of the determination process in an object detection apparatus. It is a flowchart which shows the example of the update process of the discriminator performed at the time of initialization by an object detection apparatus.

  As shown in FIG. 1, the object detection apparatus 1 according to the embodiment of the present invention includes an image acquisition unit 2, a signal processing unit 3, a storage unit 4, and an output unit 5. The object detection device 1 acquires a monitoring image obtained by imaging a predetermined monitoring space, and detects a detection target such as a person or an object captured in the image. The image acquisition unit 2, the signal processing unit 3, the storage unit 4, and the output unit 5 are connected so as to be able to transmit information to each other.

  In this embodiment, an example in which a person shown in an image is a detection target will be described. However, the present invention is not limited to this, and the present invention can also be applied to the case of detecting articles such as commodities placed in distribution, vehicles passing through, and the like.

  The image acquisition unit 2 includes a so-called surveillance camera configured to include an imaging element such as a CCD element or a C-MOS element, an optical system component, an analog / digital converter, and the like. The image acquisition unit 2 may acquire an image via an Internet network. The image acquisition unit 2 transmits the captured image to the signal processing unit 3. The interval at which the images are acquired may not be a fixed time interval.

  The image is, for example, a color image having a width of 320 pixels, a height of 240 pixels, and each pixel expressing R (red), G (green), and B (blue) with 256 gradations.

  The signal processing unit 3 includes an arithmetic circuit such as a CPU, DSP, MCU, or IC. The signal processing unit 3 is connected to the image acquisition unit 2, the storage unit 4, and the output unit 5 so as to be able to transmit information. The signal processing unit 3 includes a detection window region selection unit 30, a detection window control unit 32, a fixed classifier 34, a learning classifier 36, a classifier update unit 37, a determination unit 38, a classifier setting unit 39, and the like. By reading a program describing the processing from the storage unit 4 and executing it, the computer is caused to function as each means.

  With reference to FIG. 2, the signal processing unit 3 detects a detection target from the input image 200 obtained from the image acquisition unit 2. For convenience of explanation, the upper left corner of the image 200 is the origin X = 0, Y = 0, the horizontal direction is the X axis, the vertical direction is the Y axis, the X axis is the right direction, and the Y axis is positively increased downward. And In the input image 200, a detection object (person) is shown at the positions of the areas 203 and 204, and rectangles 201 and 202 indicate detection window areas.

  The object detection apparatus 1 scans while gradually shifting the detection window area, and determines whether a person is captured in the detection window area. The arrow indicates the upper left coordinates when the detection window area is shifted, and the detection window area is scanned so as to search the entire image 200 without omission. The rectangles in the areas 203 and 204 indicate detection window areas (person candidate areas) that are determined to be human as a result of the detection process. In the vicinity of an image region where a person is shown, there may be a plurality of detection window regions that are determined to be humans. However, the final detection window region (in FIG. (Thick line: human area). Further, a rectangle in the area 205 indicates an area where a hanger with clothes similar to a person who is a detection target is captured.

  The detection window area selection means 30 determines the width and height of the detection window area. The detection window area is scanned to determine whether or not a person is captured in the image while changing the width and height of the detection window area in order to correspond to people captured in various sizes in the image. Note that one or a plurality of widths and heights of the detection window region are stored in advance in the storage unit 4 in consideration of the size of the detection object on the image. However, even if the width and height are not stored in advance, they may be determined according to a predetermined rule.

  The detection window control means 32 determines an interval (scanning interval) for shifting the detection window region, and shifts the detection window region based on the determined scanning interval.

  The fixed discriminator 34 calculates the degree of similarity of how much the image in the detection window area resembles a person. The fixed discriminator 34 learns in advance using a large number of “person” image data and “non-person” image data.

  The similarity is calculated by the processing procedure shown in FIG. Specific examples of the fixed classifier 34, the learning classifier 36, and the classifier update means 37 shown in FIG. 1 are shown as a cascade type classifier 320, a learning classifier 330, and a classifier update means 332, respectively, in FIG.

  First, an image in the detection window region cut out from the input image is input to the cascade discriminator 320. The cascade type classifier 320 is a classifier in which a plurality of strong classifiers such as the strong classifiers 321, 322, and 323 are arranged in series (here, an example in which N strong classifiers are arranged in series is shown). ). Individual strong classifiers are Histograms of Oriented Gradients (HOG) features (Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, In Proceedings of IEEE Conference Computer Vision and Pattern Recognition 2005) To learn in advance with AdaBoost. That is, various images of people who are detection objects and images that do not show people are prepared in large quantities, each image is correctly identified as to whether it is an image of the detection object, and these data To learn with Adaboost so that both can be identified. When calculating the similarity, each strong classifier calculates the HOG feature from the input image, and calculates the similarity from the feature amount selected by Adaboost.

  The order in which the strong classifiers 321 to 323 are calculated in advance is determined, and the strong classifier 321 to be calculated first is 1, the next strong classifier 322 to be calculated is 2. Assign the same number as the order of calculation.

  Each of the strong discriminators 321 to 323 receives the image cut out as the detection window region and calculates the similarity. The strong classifiers 322, 323, etc. other than the first strong classifier 321 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). When the value is equal to or less than the threshold value (arrow F in the figure), the classifier update unit 332 updates the learning classifier 330. The threshold is set to 0, for example, and if it is greater than 0, it is similar to a person, and if it is 0 or less, it is determined that it is not similar to a person. Hereinafter, the number of strong classifiers for which the similarity is calculated is referred to as the number of determination stages.

  The learning discriminator 330 uses the similarity of each strong discriminator of the cascade discriminator 320 as an input to determine whether or not the person is a person.

  The learning discriminator 330 performs the determination only when the cascade type discriminator 320 determines that the last strong discriminator is a person (arrow T in the figure). In the example of the present embodiment, the “last strong classifier” is a strong classifier having a number represented by the value of the used stage number 44 stored in the storage unit 4. The learning discriminator 330 makes a determination only when the possibility of “person” is high in the cascade discriminator 320, that is, when the similarity obtained by the strong discriminator at the final stage is larger than the threshold (arrow T in the figure). Do. The learning discriminator 330 receives the similarity Sn (n is 1 to M: M is the number of determination stages (1 ≦ M ≦ N)) obtained by the strong classifiers up to the number of determination stages in the cascade classifier 320 as an input. A similarity vector S = (similarity of the first strong classifier, similarity of the second strong classifier,..., A vector representation of the similarities obtained by the strong classifiers of the cascade type classifier 320 arranged side by side. (Similarity of Mth strong classifier) = (S1, S2,..., SM).

  Calculating the probability P (person | S) of “person” when the similarity vector S is obtained, and the probability P (non-person | S) of “non-person” when the similarity S is obtained; If P (person | S)> P (non-person | S), it is determined as “person”, otherwise it is determined as “non-person”.

Calculations of P (person | S) and P (non-person | S) can be obtained by the formulas (1) and (2) from Bayes' theorem.

  Here, P (S | People) obtains the probability (likelihood) of obtaining the similarity S in the case of “People”, and P (S | People) obtains the similarity S in the case of “Non-People”. Probability (likelihood). P (person) is a prior probability that indicates the probability of “person” being included in the detection window area, and similarly P (other than person) is included in the detection window area with a probability of “non-person”. It is a prior probability that represents whether or not. Since the probability that “person” and “non-person” are included in the detection window region in advance is not known, it is assumed that the prior probability P (person) = P (non-person) = 0.5. Since P (non-person | S) = 1−P (person | S), if P (person | S) is obtained, P (non-person | S) is easily obtained. Hereinafter, P (person | S) is referred to as the similarity in the learning discriminator 330.

When a random variable vector s having the similarity of each strong classifier as an element is prepared, the distribution P (s | person) of the similarity vector s of “person” and the distribution P (s) of the similarity vector s of “non-person”. If (other than people) is obtained, P (people | s) can be calculated. Although the similarity vector s is multi-dimensional data, assuming that the similarity sn of each strong classifier is independent, P (s | people) and P (s | others) are expressed by equations (3) and (4). Desired.

  Here, P (sm | person) is the distribution of similarity of “m” of the mth strong classifier, and P (sm | other) is the similarity of “mother” of the mth strong classifier. Distribution.

  In the present embodiment, the distribution of similarity of “person” and the distribution of similarity of “non-person” are expressed by a histogram such that the sum of the distributions of similarity of all bins (separators) is 1. Each histogram is stored in the similarity histogram 42 of the storage unit 4. However, the similarity distribution is not limited to the histogram, and may be expressed by a normal distribution, a mixed normal distribution, or the like.

  The “person” similarity histogram 42 is configured by obtaining a similarity distribution of each strong classifier using learning data that is an image including “person” prepared in advance. In the present embodiment, the “person” similarity histogram 42 is not sequentially learned but may be updated. The similarity histogram 42 of “non-person” assumes that no person is included in the first few frames when the surveillance camera is installed, and the similarity distribution of the strong classifiers obtained at that time is shown. Configure by asking. Thereafter, when the detection window region is determined to be “non-human” in the cascade type discriminator 320 or the learning discriminator 330, the similarity histogram 42 of “non-human” is used by using the similarity obtained by the cascade type discriminator 320. Are updated sequentially.

  FIG. 4 shows an example of the similarity histogram 42 when the number of strong classifiers in the cascade classifier 320 is N = 2. The similarity histogram 401 is a distribution of similarity “person” 402: P (s1 | person) and similarity distribution 403: P (s1 | person) of “non-person” in the similarity s1 of the first strong classifier. Except). The similarity histogram 411 shows the distribution of similarity of “person” 412: P (s2 | person) and the distribution of similarity of “non-person” 413: P (s2 | person) in the similarity s2 of the second strong classifier Except). The four similarity histograms 402, 403, 412, and 413 are normalized so that the sum of the similarity distributions of all bins is 1.

  When the determination is made only with the cascade classifier 320, if the degree of similarity obtained by each strong classifier is larger than a predetermined threshold (for example, set to 0), it is determined as “person”. Then, the probability is calculated from the obtained similarity of each strong classifier and the past similarity histogram 42 and can be adaptively determined.

  As an example, when the similarity distribution of “person” and “non-person” is represented by the similarity histograms 401 and 411 shown in FIG. 4, the similarity of the first strong classifier is S1 = 1.0, A case where the similarity of the second-stage strong classifier is S2 = 1.0 will be described. When the threshold values in the first and second stage strong classifiers are 0, the similarity exceeds the threshold value in both the first and second stage strong classifiers. It is determined as “person”. On the other hand, in the learning discriminator 330, the probability of “person” when the similarity vector S = (1.0, 1.0) is obtained: P (person | S = (1.0, 1.0) ) = P (S1 = 1.0 | People) × P (S2 = 1.0 | People) × P (People) / A and the probability of “Non-People”: P (Non-People | S = (1. 0, 1.0)) = 1−P (person | S = (1.0, 1.0)). However, A = P (S | person) × P (person) + P (S | other than person) × P (other than person). Based on the histograms 401 and 411 shown in FIG. 4, P (person | S = (1.0,1.0)) <P (non-person | S = (1.0,1.0)). The discriminator 330 determines “other than human”.

  If a similarity histogram matching the characteristics of the installation environment of the object detection device can be constructed, the learning discriminator 330 can make a more accurate determination based on the similarity calculation result from the cascade discriminator 320.

  After the determination by the learning discriminator 330, the learning discriminator 330 is updated by the discriminator update unit 332. Further, the similarity management unit 334 associates the similarity, the size (width and height) of the detection window region, and the center coordinates, and stores them in the storage unit 4 as the determination information history 40.

  The discriminator update unit 332 updates the similarity histogram of the learning discriminator 330 according to the determination result. In the present embodiment, the learning classifier 330 updates the learning classifier 330 when the similarity obtained by each strong classifier is equal to or less than a threshold value (arrow F in FIG. 3). Further, even when the similarity obtained by the strong discriminator in the cascade discriminator 320 exceeds the threshold value (arrow T in FIG. 3), the learning discriminator 330 determines “other than human” (in FIG. 3). The learning discriminator 330 is also updated in the case of F). That is, only the “non-person” similarity histogram 42 is updated, and the “person” similarity histogram 42 is not updated. Further, the number of used stages indicating how many strong classifiers of the cascade classifier 320 are used is updated from the determination results of the cascade classifier 320 and the learning classifier 330. The updated used stage number is stored in the storage unit 4 as the used stage number 44.

  The discriminator update unit 332 updates the similarity histogram 42 obtained up to one time ago with the data obtained at the current time. For example, as shown in FIG. 5, α blending is used for the update process by the discriminator update unit 332. FIG. 5 shows a process of updating the n-th “non-human” similarity histogram 42 of the learning discriminator 330, based on the similarity histogram 501 before the update up to the previous time and the data obtained at the current time. An updated similarity histogram 502 is shown.

  When it is determined whether or not an image of a person is included in the detection window area, it is determined that the cascade discriminator 320 determines “person” and the learning discriminator 330 determines “other than human”. A method of updating the similarity histogram 501 corresponding to the nth strong classifier when the similarity of the nth strong classifier is 0.8 is shown. First, each bin of the similarity histogram 501 before update is multiplied by (1.0−α). Thereafter, a weight corresponding to α is added to the bin corresponding to the similarity of 0.8 obtained at the current time (the hatched portion of the bin 503 in FIG. 5). α is a predetermined parameter. When α is large, the influence of past information is reduced earlier, and when α is small, the influence of past information is longer. When updating for the first time, that is, when the similarity histogram 501 is empty, the bin value corresponding to the obtained data is set to 1. However, since the influence of the first obtained data is large in this way, α may be set to a relatively large value while the number of updates is small, and may be set to a smaller value as the number of updates increases. Good.

  The discriminator update unit 332 also updates the number of stages used by the cascade discriminator 320 according to the determination results of the cascade discriminator 320 and the learning discriminator 330. In the present embodiment, the number of stages used in the cascade discriminator 320 is changed according to the policy shown in [Example 1] below.

[Example 1]
When the determination result of the cascade discriminator 320 and the determination result of the learning discriminator 330 are different from each other for a certain period, the value of the used stage number 44 is increased by 1. In this example, the value of the number of used stages 44 is increased by 1. However, the present invention is not limited to this, and any number of stages that can be increased may be determined.
The number of the used stage number 44 that is currently set in the cascade classifier 320 is not continuously used (that is, the judgment stage number of the cascade type classifier 320 is continuously less than the used stage number 44). The number of used stages 44 is decreased by 1 when the number of times exceeds a predetermined threshold. However, the number by which the set value of the used stage number 44 is decreased is not limited to 1 and may be arbitrarily determined.

  Note that the initial value of the number of used stages 44 is, for example, the maximum number of stages of the strong classifier used when the input image for several frames is processed at the initial setting immediately after the camera is installed.

  Here, referring back to FIG. 1, the determination means 38 uses the size (width, height) and center coordinates of the detection window region stored as the determination information history 40, and the similarity of the learning discriminator 330. Finally, determine where the person is in the image. In the data included in the determination information history 40, a detection window region whose similarity of the learning discriminator 330 is equal to or higher than a threshold (for example, set to 0.8) is extracted as a human candidate region. If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. The detection window region having the highest similarity of the learning discriminator 330 is selected as the human region among the grouped human candidate regions. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5.

  The discriminator setting unit 39 uses the discriminator corresponding to the position of the detection window region set by the detection window control unit 32 in the process of determining whether or not a human image is included in the detection window region. Set as. For example, the discriminator setting means 39 sets the used stage number 44 of the cascade discriminator 320 and the “non-human” similarity histogram 42 of the learning discriminator 330 according to the position coordinates of the detection window region. Similarity calculation processing of the cascade discriminator 320 with respect to the image of the detection window area is performed using the number of used stages 44 set by the discriminator setting means 39. Similarly, the similarity calculation process of the learning discriminator 330 for the image in the detection area is performed using the “non-human” similarity histogram 42 set by the discriminator setting means 39.

  Taking the image 211 in FIG. 2 as an example, the setting of the classifier corresponding to the detection window region will be described. An image 211 shown in FIG. 2 corresponds to the input image 200, and the input image 200 is divided into divided areas 1, 2,. The position coordinates and size of each divided area are stored in the storage unit 4 in advance. Further, the storage unit 4 stores, for each of the divided areas 1, 2,..., 12, the number of used stages 44 of the cascade type classifier 320 and the similarity histogram 42 of “non-human” of the learning classifier 330. The discriminator setting means 39 reads out from the storage unit 4 the similarity histogram 42 and the number of used steps 44 of “non-human” in the divided area corresponding to the position coordinates of the detection window area. The cascade discriminator 320 and the learning discriminator 330 perform each process by using the number of used stages 44 and the “non-human” similarity histogram 42 read from the storage unit 4 by the discriminator setting unit 39. In the present embodiment, the same “person” similarity histogram 42 is used for all divided regions.

  Also in the update process by the discriminator update means 332, the similarity histogram 42 and the number of used stages 44 of the divided areas corresponding to the position coordinates of the detection window area are updated. As a result, a learning discriminator 330 for each divided region of the input image 200 is constructed using the similarity histogram 42 of “non-human” of the divided region corresponding to the position of the detection window region, and further, cascade identification is performed for each divided region. The number of used stages 44 of the device 320 is set. Therefore, the cascade discriminator 320 and the learning discriminator 330 can be optimized according to the divided areas.

  In addition, the aspect which divides | segments the input image 200 is not restricted to the aspect illustrated to the image 211 of FIG. For example, the number of divided areas may be increased or decreased as compared with the image 211 in FIG. 2, and the shape of each divided area may not be a rectangular area. Further, for example, all the divided regions may not have the same size.

  The discriminator setting unit 39 may set the discriminator in consideration of not only the position of the detection window region but also the size (width, height) of the detection window. For example, a divided region obtained by dividing the input image 200 in a different manner depending on the size of the detection window is set in advance, and a similarity histogram 42 of “non-human” is obtained for each combination of the size of the detection window and each divided region. The number of used stages 44 is stored in the storage unit 4. The discriminator setting means 39 includes a similarity histogram 42 of “non-human” corresponding to a combination of the size of the detection window determined by the detection window area selection means 30 and the divided area corresponding to the position coordinates of the detection window area, and the number of stages used. 44 is read from the storage unit 4. Then, each processing of the cascade type discriminator 320 and the learning discriminator 330 is performed using the read used stage number 44 and the similarity histogram 42 of “non-human”.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the similarity management unit 334, and stores, as the determination information history 40, three pieces of information including the size (width and height) of the detection window region, the center coordinates, and the similarity of the learning discriminator 330 in association with each other. To do. For each strong classifier included in the cascade classifier 320, a similarity histogram 42 of “person” and “non-person” is stored in association with the number of each strong classifier. Further, the storage unit 4 stores the current number of used stages 44 of the cascade discriminator 320. Note that the similarity histogram 42 and the number of used stages 44 are stored for each divided region into which the input image 200 is divided.

  The output unit 5 can be configured to include a sound output unit that outputs a notification sound and a display unit that displays an input image. When the determination unit 38 detects a human area, an alarm is sounded by a sound output unit such as a speaker or a buzzer, or an input image is displayed on an external display device such as a display. The output unit 5 may include an interface for connecting the computer to a network or a telephone line. In this case, the output unit 5 sends an input image and human area information to a center device (not shown) via information transmission means such as a telephone line or the Internet. The center device is a host computer installed in a center or the like that monitors a detection target in an image.

  Hereinafter, the process in the object detection apparatus 1 will be described with reference to the flowchart of FIG.

  In step S <b> 10, an image is acquired by the image acquisition unit 2, and the image is transmitted to the signal processing unit 3. The image acquisition timing is a fixed time interval.

  In step S20, the size (width and height) of the detection window region is determined. The entire image is scanned with each size of the detection window region while sequentially changing the size of the detection window region to a plurality of preset sizes. In this embodiment, since the detection window region is rectangular, it is only necessary to determine the width and height as the size. However, the detection window region may have an arbitrary shape, and in that case, the shape and size are determined. To decide. This processing is performed by the detection window region selecting means 30.

  The processing from step S30 to step S50 is repeated until the entire image has been scanned with the size of the detection window region set in step S20.

  In step S30, the position of the detection window area is determined. This process is performed by the detection window control means 32. This process is performed according to the flowchart shown in FIG. The upper left coordinate of the detection window area is called a start point, the start point of the detection window area to be determined is (SX, SY), and the previous start point of the detection window area is (BX, BY).

  Referring to FIG. 7, in step S301, it is determined whether or not the size of the detection window region has been changed. If the size of the detection window area has just been changed, the start point (SX, SY) of the detection window area is set to (0, 0) in step S305, and the process proceeds to step S40. If it is not the first process after the size of the detection window area is changed, the process proceeds to step S302.

  In step S302, the start point (BX, BY) of the previous detection window area is read. In step S303, it is determined whether scanning has been performed from the start point (BX, BY) of the previous detection window area read in step S302 to the right end of the image. When scanning to the right edge of the image is completed in the detection window area, that is, when the right edge BX + W (W is the width of the detection window area) of the detection window area coincides with the right edge of the image, the detection window area in step S306 Is set to (SX, SY) = (0, BY + q), and the process proceeds to step S40. However, q is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, q = 4 pixels is set. If not scanned to the right end, the process proceeds to step S304.

  In step S304, the detection window region start point (SX, SY) is determined so as to shift the detection window region by p pixels in the X direction and not in the Y direction. That is, the detection window region start point (SX, SY) = (BX + p, BY). However, p is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, p = 4 pixels is set. Thereafter, the process proceeds to step S35 (FIG. 6).

  In step S35, the classifier setting unit 39 sets a classifier corresponding to the position of the detection window region. In the present embodiment, the “non-human” similarity histogram 42 and the number of used steps 44 stored in the storage unit 4 for the divided areas corresponding to the position coordinates of the detection window area among the divided areas obtained by dividing the input image 200 in advance. Read from the storage unit 4. The similarity histogram 42 of “people” is read from the storage unit 4 regardless of the position coordinates of the detection window region.

  In the following description, when describing the similarity histogram 42 and the number of used stages 44 of “non-human”, the similarity of “non-human” with respect to the divided area corresponding to the position coordinates of the current detection window area unless otherwise specified. The degree histogram 42 and the number of used stages 44 are indicated.

  In step S40, a similarity indicating how much the image in the detection window region set in step S30 resembles a person is obtained. This is the discrimination processing of the cascade type discriminator 320 and the learning discriminator 330 shown in FIG. The identification process will be described with reference to the flowchart of FIG.

  In step S401, the HOG feature amount is calculated from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  Steps S <b> 402 to S <b> 405 are processing of the cascade discriminator 320. In step S402, it is determined whether or not the number of strong discriminators for the number of stages represented by the value of the used stage number 44 has been investigated. When the investigation is completed with the strong classifiers for the number of used stages 44, the process proceeds to step S406, and when not completed, the process proceeds to step S403, and the next strong classifier is investigated. In step S403, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S404, the similarity calculated in step S403 is stored in the temporary storage area together with the number of the strong classifier that has been determined. This is used in similarity calculation in the learning discriminator 330 in step S407 and update processing in step S50.

  In step S405, it is determined whether the similarity calculated in step S403 is greater than a preset threshold value. If it is larger than the threshold value, the process proceeds to step S402. If it is equal to or smaller than the threshold value, the similarity of a learning discriminator 330 described later is stored as being equal to or smaller than the threshold value in step S501, and the process proceeds to step S50.

  Steps S406 to S408 are processing of the learning discriminator 330. In step S406, among the similarity histograms 42 of the strong classifiers up to the number of determination stages of the cascade classifier 320, the similarity histogram 42 of “non-human” whose update count is equal to or greater than the reference value is extracted. This is because if the similarity histogram 42 with a small number of updates is used, the reliability of the similarity calculated by the learning discriminator 330 becomes low. Therefore, the similarity in the learning discriminator 330 is calculated using the similarity histogram 42 updated more than the reference value and the corresponding similarity of the strong discriminator stored in step S404.

  In step S407, the learning discriminator 330 calculates the similarity. The similarity is calculated using P (person | S) in the formula (1). In step S408, the similarity calculated in step S407 is stored in the storage unit 4 as the determination information history 40 in association with the size (width and height) of the detection window region and the center coordinates, and the process proceeds to step S50.

  In step S50, the similarity histogram 42 and the number of used stages 44 are updated. This is done by the discriminator update means 37. The update process is performed according to the flowchart of FIG. Steps S501 to S502 are the updating process of the similarity histogram 42, and the processes after step S504 are the updating process of the number of used stages 44. In the present embodiment, in the update process of the similarity histogram 42, the similarity histogram 42 of “non-person” is updated, and the similarity histogram 42 of “person” is not updated.

  In step S501, if the similarity calculated by the learning discriminator 330 in step S407 is larger than the threshold value, it is highly likely that the person is “person”, so the similarity histogram 42 is not updated, and the process proceeds to step S508. .

  In step S502, the “non-human” similarity histogram 42 corresponding to each strong classifier included in the cascade classifier 320 is updated with the similarity stored in step S404 of FIG. Accordingly, only the similarity histogram 42 corresponding to the strong classifier used for the current determination is updated. Here, the similarity histogram 42 of “non-human” is updated. The update method is as described above for the classifier update means 332. After the update process, the process proceeds to step S504.

  In step S504, the number of times that the strong classifier corresponding to the currently used number of used stages 44 is continuously unused (hereinafter referred to as “continuous unused number of times”) is checked. If the number of consecutive unused times is larger than the preset threshold value Tun times, the set value of the number of used stages 44 is decreased by 1 in step S509. If it is equal to or less than the threshold value Tun, the process proceeds to step S505. In the present embodiment, the number of used stages 44 is decreased by “1” based on the number of consecutive times that the strong classifier corresponding to the currently set used stage number 44 is unused, but is currently set. The set value of the used stage number 44 may be decreased by the number of strong classifiers in which the number of unused individual classifiers in each stage up to the used stage number 44 is larger than the threshold Tun times.

  If it is determined in step S505 that the number of determinations is greater than the threshold Tjc, the number of mismatches is greater than the threshold Ter, and the value of the current used stage number 44 is not the maximum number of the strong classifier stored in the storage unit 4 in advance, In S506, the set value of the number of used stages 44 is increased by 1. Here, the number of times of determination represents the number of times that the determination processing is performed by the cascade type discriminator 320 after the use stage number 44 is updated. Further, the number of mismatches is the number of times that the determination result of the cascade discriminator 320 and the determination result of the learning discriminator 330 do not match.

  The number of mismatches may be, for example, the number of times “mismatch” is made according to the criteria shown in the following [Example 2] in the determination process performed by the cascade type discriminator 320 after the use stage number 44 is updated.

[Example 2]
-Cascade type discriminator 320 determines "other than person" → matches (In this embodiment, when cascade type discriminator 320 determines "other than human", learning discriminator 330 does not make a determination, but in this case Is a match.)
-Cascade discriminator 320 determines "person" and learning discriminator 330 determines "person" → matches-Cascade discriminator 320 determines "person" and learning discriminator 330 determines "other than human" → Disagreement

  Note that the values of the continuous unused count, the determination count, and the mismatch count are stored in the storage unit 4 in association with the number of used steps 44 of the divided region corresponding to the current position coordinates of the detection window region, and the discriminator update means 37 Reads out and changes (described later) each value with reference to the storage unit 4. If the condition is not satisfied in step S505, the process proceeds to step S508.

  In step S507, after the value of the number of used stages 44 is updated in step S506 or step S509, initialization is performed to set the number of determinations, the number of consecutive unused times, and the number of mismatches to 0, respectively, and the process proceeds to step S60.

  Step S508 is processing performed when the value of the number of used stages 44 is not updated. In step S508, the determination count is incremented by 1, and the continuous unused count is initialized (set to 0) if the determination step count and the use step count 44 match, and if not, it is incremented by one. In addition, if the determination results of the current cascade type discriminator 320 and the learning discriminator 330 do not match, the number of mismatches is incremented by 1. Otherwise, the value of the number of mismatches is not changed. After step S508, the process proceeds to step S60.

  Referring to FIG. 6 again, in step S60, it is determined whether or not the detection process has been completed for detection window regions of all sizes. If the inspection is completed for all detection window regions, the process proceeds to step S70, and if not, the process returns to step S20.

  In step S70, a person is finally captured from the size (width and height) of each detection window region stored in the determination information history 40 obtained in step S50, the center coordinates, and the similarity of the learning discriminator 330. Determine the position. This processing is performed by the determination means 38. The determination process is performed according to the flowchart of FIG.

  The processing of steps S701 to S702 is performed for all detection window regions stored as the determination information history 40 obtained in step S50. In step S701, it is determined whether or not the similarity of the learning discriminator 330 obtained by the learning discriminator 330 is greater than a threshold value. If the condition is satisfied, the process proceeds to step S702. Otherwise, the next detection is performed. Judge the window area. In step S702, the detection window area that satisfies the condition in step S701 is added to the human candidate area.

  As shown in FIG. 2, candidate candidate areas obtained by repeating steps S701 to S702 on the information stored in the determination information history 40 are areas 203 and 204 near the area where the person is shown. May be extracted multiple times. In steps S703 to S705, a human region is finally selected from a plurality of human candidate regions.

  In step S703, using the size and center coordinates of the detection window area extracted as the human candidate area, the human candidate areas that overlap a certain area (for example, more than half the area of the detection window area) are grouped together. . In step S704, the detection window region having the highest similarity of the learning discriminator 330 is selected as a human region among the human candidate regions for each group created in step S703. In step S705, the detection window area selected in step S704 is set as a human area, and the process proceeds to step S80.

  In step S80, if there is at least one human area based on the human area determined in step S70, the detection window information of the determined human area is output as an abnormal signal together with the image. This processing is performed by the output unit 5.

  The initial setting process when the surveillance camera is installed will be described below. At the time of initial setting, on the assumption that there is no person in the imaging area of the surveillance camera, for each divided area of the input image, the similarity histogram 42 of “non-person” corresponding to each strong classifier 42 and the number of stages used 44 are used. Register the initial value of. It is assumed that the initial setting process is completed after a predetermined number of images are processed after the monitoring camera is installed.

  The initial setting process is a process in which steps S40 and S50 are slightly changed except for steps S70 and S80 in the flowchart of the detection process in FIG. In addition, only differences between the processes of steps S40 and S50 will be described.

  In step S40, the process from step S401 to S405 in FIG. 8 is performed. If the determination result in step S402 is YES, the process proceeds to step S50 without performing the process in steps S406 to S408. Since the number of used stages 44 is not determined at the time of initial setting, in step S402, the maximum number of usable strong classifiers (ie, the maximum number of strong classifiers) stored in advance in the storage unit 4 is used. Is used as a determination process.

  The discriminator update process (step S50) at the time of initial setting will be described with reference to FIG.

  In step S551, the similarity histogram 42 of “non-human” of the corresponding strong classifier is updated using the similarity of the strong classifier up to the number of determination stages obtained in step S40. After the update process, the process proceeds to step S552.

  In step S552, it is checked whether or not the current process is the first determination process in the cascade discriminator 320. If it is the first determination process, the current determination stage number is stored as the used stage number 44 in step S554. Store in part 4. If it is not the first determination, the process proceeds to step S553.

  In step S553, if the current determination step number is larger than the value of the used step number 44 stored in the storage unit 4, the current determination step number is stored in the storage unit 4 as the use step number 44 in step S554. If the current determination step number is less than or equal to the stored use step number 44 value, the process proceeds to step S60.

  By the processing of steps S552 to S554, the maximum value of the number of stages of the strong classifier used in the cascade type classifier 320 for which the initial setting has been completed is set as the initial value of the used stage number 44.

  In the above-described embodiment, the mode in which the function of each unit of the object detection device 1 is realized by one computer has been described, but the present invention is not limited to this. The functions of each part of the object detection device 1 can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by a single computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Image acquisition part, 3 Signal processing part, 4 Memory | storage part, 5 Output part, 30 Detection window area | region selection means, 32 Detection window control means, 34 Fixed discriminator, 36 Learning discriminator, 37 Discriminator update Means, 38 judging means, 39 discriminator setting means, 40 judgment information history, 42 similarity histogram, 44 number of stages used, 310 detection window cutting means, 320 cascade type discriminator, 321, 322, 323 strong discriminator, 330 learning Classifier, 332 Classifier update means, 334 similarity management means.

Claims (4)

An object detection device for detecting a detection target from an image obtained by photographing a monitoring area,
An image acquisition unit for acquiring the image;
A detection window region selection means for capturing a detection window region image from detection windows sequentially set in the image input from the image acquisition unit;
A discriminator setting means for setting a discriminator corresponding to the set position of the detection window;
The discriminator having a cascade type discriminator in which a plurality of strong discriminators that determine whether or not a detection target exists in the detection window region image is connected in series .
A discriminator updating unit configured to update the discriminator set by the discriminator setting unit by reducing the number of used stages of the cascade discriminator when there is a stage not using the strong discriminator ;
An object detection device characterized by comprising: The discriminator includes a learning discriminator that determines whether or not a detection target exists in the detection window region image using information learned according to a set position of the detection window from an image obtained by photographing the monitoring region. It has in the latter part of the cascade type discriminator,
The identifier update unit, according to claim 1, characterized in that increasing the number of used stages of the cascade classifier when the judgment result of the cascade classifier and a determination result of the learning classifier does not match Object detection device. The discriminator, determining learning classifier whether the detection target in the detection window region image exists by using the information learned in accordance with the set position of the detection window from images taken the monitoring area It has in the latter part of the cascade type discriminator ,
2. The object detection apparatus according to claim 1, wherein the classifier update unit performs learning of the learning classifier using the detection window region image. The setting position of the detection window set sequentially is associated with any of the divided areas obtained by dividing the image into a plurality of areas in advance.
The identifier setting means, the object according to any one of the detection window claims 1 to to and sets the same the classifier for each of the divided areas corresponding to the set position of the 3 Detection device.

Images