JP2010140315A - Object detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a processing load to prevent detection leakage, in object detection from an image. <P>SOLUTION: An object detection device includes: an image acquisition part 2 for acquiring the image; an identification device 34 for fetching a detection window area image set with a detection window from the image input from the image acquisition part 2, and outputting decision of whether or not a detection object is present in the detection window area image and reliability showing a degree of certainty of the presence of the detection object in the decision; a detection window control means 32 for setting a subsequent scanning spacing of the detection window to be shorter as the reliability becomes higher; and a decision means 36 deciding whether or not the detection object is present based on the output from the identification device. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an object detection apparatus that detects a detection target from an image.

  An object detection device that detects whether an image area corresponding to a detection target is included in an image is known.

  Patent Document 1 discloses a template matching technique that performs matching while sequentially shifting a template to be identified with respect to a scanning region on an image. Matching processing is performed by scanning the detection window region with a large scanning interval, and matching is performed on eight neighboring regions at a level where there are more calculation target pixels than the current when the similarity to the template to be identified is high. Therefore, the matching process is speeded up and accuracy is improved.

  Patent Document 2 discloses an object detection device that detects an object by sequentially scanning an input image using a rectangular area as a detection window area. The similarity with the object is calculated in the detection window region. If the similarity is sufficiently high, the scanning interval of the detection window region is widened. If the similarity is not sufficiently high, the scanning interval is shortened.

  There has also been proposed an object detection apparatus that inputs an image to a discriminator that has previously learned whether or not the target object exists and detects the target object in the image. For example, Patent Document 3 discloses a person tracking apparatus that tracks a person using a particle filter, and uses a cascade type discriminator and uses the number of passing stages of the cascade as an indicator of humanity.

JP-A-2-159682 Japanese Patent No. 3517518 JP 2008-26974 A

  By the way, in order to obtain the position where the detection target exists from the image with higher accuracy, it is necessary to perform matching while shifting the detection window region at a fine scanning interval. However, if the scanning interval is reduced, the processing load increases, and it takes time until the detection target is detected. On the other hand, if matching is performed while shifting the detection window region at a large scanning interval, so-called detection omission that cannot be detected despite the presence of a detection target occurs.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an object detection device that reduces processing load and eliminates detection omissions.

  One aspect of the present invention is an object detection device that detects a detection target from an image, and an image acquisition unit that acquires the image, and a detection window is set from the image input from the image acquisition unit. A detection window region image, a determination as to whether or not the detection target exists in the detection window region image, and a reliability indicating a degree of certainty of the detection target in the determination. It is determined whether or not the detection target exists based on the output from the discriminator, the detection window control means for setting the scanning interval of the subsequent detection window to be shorter as the reliability becomes higher, and the output from the discriminator. And determining means.

  Here, it is preferable that the detection window control means sets the scanning interval within a range that does not exceed the size of the detection window.

  The discriminator outputs a degree of similarity indicating a degree of similarity between the detection window region image and the detection target, and the determination unit has the reliability equal to or higher than a predetermined value and overlaps each other. It is preferable that the detection window region image having a high similarity among the detection window region images is determined as a detection target image including an image of the detection target.

  The discriminator includes a cascade type discriminator in which a plurality of strong discriminators are connected in series, and each of the plurality of strong discriminators connected in series includes the detection object in the detection window region image. The number of stages from the first stage of the strong classifier of the last stage among the strong classifiers that output the identification result that the detection target exists in the cascade type classifier is output. The reliability is preferably set. In addition, when using the output of the identification result that the detection target does not exist, the same is true for the number of stages preceding the strong classifier that outputs the identification result.

  According to the present invention, it is possible to reduce the processing load in the object detection apparatus and eliminate detection omissions.

<First Embodiment>
As shown in FIG. 1, the object detection apparatus 1 according to the embodiment of the present invention includes an image acquisition unit 2, a signal processing unit 3, a storage unit 4, and an output unit 5. The object detection device 1 acquires a monitoring image obtained by imaging the monitoring space, and detects a detection target such as a person or an object captured in the image. The image acquisition unit 2, the signal processing unit 3, the storage unit 4, and the output unit 5 are connected so as to be able to transmit information to each other.

  In this embodiment, an example in which a person captured in an image is detected will be described. However, the present invention is not limited to this, and the present invention can also be applied to the case of detecting articles such as commodities placed in distribution, vehicles passing through, and the like.

  The image acquisition unit 2 includes a so-called surveillance camera configured to include an imaging element such as a CCD element or a C-MOS element, an optical system component, an analog / digital converter, and the like. The image acquisition unit 2 may acquire an image via an Internet network. The image acquisition unit 2 transmits the captured image to the signal processing unit 3. The interval at which the images are acquired may not be a fixed time interval. Further, the acquired image may not be an image of a fixed place.

  The image is, for example, a color image having a width of 320 pixels, a height of 240 pixels, and each pixel representing R (red), G (green), and B (blue) with 256 gradations.

  The signal processing unit 3 includes an arithmetic circuit such as a CPU, DSP, MCU, or IC. The signal processing unit 3 is connected to the image acquisition unit 2, the storage unit 4, and the output unit 5 so as to be able to transmit information. The signal processing unit 3 reads the program describing the processing in each unit such as the detection window region selection unit 30, the detection window control unit 32, the discriminator 34, and the determination unit 36 from the storage unit 4 and executes the computer. It functions as each means.

  As shown in the upper diagram of FIG. 2, the signal processing unit 3 detects a detection target from the input image 200 from the image acquisition unit 2. For convenience of explanation, the upper left corner of the image 200 is the origin X = 0, Y = 0, the horizontal direction is the X axis, the vertical direction is the Y axis, the X axis is the right direction, and the Y axis is positively increased downward. And In the input image 200, a detection object (person) is shown at the positions of the areas 203 and 204, and rectangles 201 and 202 indicate detection window areas.

  The object detection apparatus 1 scans while gradually shifting the detection window area, and determines whether a person is captured in the detection window area. The arrow indicates the upper left coordinates when the detection window area is shifted, and the detection window area is scanned so as to search the entire image 200 without omission. The rectangles in the areas 203 and 204 indicate detection window areas (person candidate areas) that are determined to be human as a result of the detection process. In the vicinity of an image region where a person is shown, there may be a plurality of detection window regions that are determined to be humans. However, the final detection window region (in FIG. (Thick line: human area).

  The detection window area selection means 30 determines the width and height of the detection window area. The detection window area is scanned to determine whether or not a person is captured in the image while changing the width and height of the detection window area in order to correspond to people captured in various sizes in the image. Note that one or a plurality of widths and heights of the detection window region are stored in advance in the storage unit 4 in consideration of the size of the detection object on the image. However, even if the width and height are not stored in advance, they may be determined according to a predetermined rule.

  The detection window control means 32 determines an interval (scanning interval) for shifting the detection window region, and shifts the detection window region based on the determined scanning interval.

  The discriminator 34 calculates the degree of similarity to how much the image in the detection window region resembles a person. The similarity is calculated by the processing procedure shown in FIG. First, a detection window region is extracted from the input image by the detection window extraction means 310 and input to the cascade discriminator 320. The cascade type discriminator 320 is a discriminator in which a plurality of strong discriminators such as strong discriminators 321, 322 and 323 are arranged in series (here, an example in which N discriminators are arranged in series) is shown. . Individual classifiers have Histograms of Oriented Gradients (HOG) features (Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, In Proceedings of IEEE Conference Computer Vision and Pattern Recognition 2005) Used to learn in advance with AdaBoost. In other words, a large number of images of various standing states of people who are detection objects and images in which people are not captured are prepared, and whether each image is an image of the detection object is correctly identified. Using these data, Adaboost is trained so that both can be identified. When calculating the similarity, each discriminator calculates the HOG feature from the input image, and calculates the similarity from the feature amount selected by Adaboost.

  The order in which the strong classifiers 321 to 323 are calculated in advance is determined. The strong classifier 321 to be calculated first is 1, the next strong classifier 322 to be calculated is 2. Assign the same number as the order of

  Each of the classifiers 321 to 323 receives the image cut out as the detection window region and calculates the similarity. The discriminators 322, 323, etc. other than the earliest discriminator 321 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). If it is equal to or less than the threshold value (arrow F in the figure), the similarity and reliability are output to the similarity / reliability management means 330, and the determination information history 40 together with the size (width, height) and center coordinates of the detection window area. Record as.

  The threshold is set to 0, for example, and if it is greater than 0, it is similar to a person, and if it is 0 or less, it is determined that it is not similar to a person. The reliability is the degree of certainty that the detection target exists in the image cut out as the detection window area, and is used when setting the scanning interval of the detection window area. For example, the strong classifier number (number of stages from the first stage) of the last class among the strong classifiers that output the discrimination result that the detection target exists (T in the figure) in the cascade classifier 320 is used. When the determination is made by the strong classifier 323 at the final stage, the similarity, the reliability, and the case of being larger than the threshold (T arrow in the figure) and below the threshold (F arrow in the figure) The size (width, height) of the detection window region and the center coordinates are output to the similarity / reliability management means 330 and recorded as the determination information history 40.

  The determination means 36 finally uses the size (width, height) of the detection window area and the center coordinates, similarity and reliability (strong classifier identification number) stored as the determination information history 40. Determine where people are in the image.

  From the data included in the determination information history 40, data whose reliability coincides with the identification number of the last stage of the strong classifier included in the cascade classifier 320 and whose similarity is larger than the threshold is extracted, and the human candidate area And If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. The detection window region having the highest similarity among the collected human candidate regions is selected as the human region. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the similarity / reliability management unit 330, and associates four types of information, that is, the size (width, height) of the detection window area, the center coordinates, the similarity, and the reliability as the determination information history 40. Remember. The determination information history 40 is read and written as necessary by the detection window control means 32, the discriminator 34, and the determination means 36.

  The output unit 5 can be configured to include a sound output unit that outputs a notification sound and a display unit that displays an input image. When the determination unit 36 detects a human area, an alarm is sounded by a sound output unit such as a speaker or a buzzer, or an input image is displayed on an external display device such as a display. The output unit 5 may include an interface for connecting the computer to a network or a telephone line. In this case, the output unit 5 sends an input image and human area information to a center device (not shown) via information transmission means such as a telephone line or the Internet. The center device is a host computer installed in a center or the like that monitors a detection target in an image.

  Hereinafter, processing in the object detection apparatus 1 will be described with reference to the functional block diagram of FIG. 1 and the flowcharts of FIGS.

  In step S <b> 10, an image is acquired by the image acquisition unit 2, and the image is transmitted to the signal processing unit 3. The image acquisition timing may be a fixed time interval or may be a timing when there is some request from the outside.

  In step S20, the size (width and height) of the detection window region is determined. The entire image is scanned with each size of the detection window region while sequentially changing the size of the detection window region to a plurality of preset sizes. In this embodiment, since the detection window region is rectangular, it is only necessary to determine the width and height as the size. However, the detection window region may have an arbitrary shape, and in that case, the shape and size are determined. To decide. This processing is performed by the detection window region selecting means 30.

  The processing from step S30 to step S50 is repeated until the entire image has been scanned with the size of the detection window region set in step S20.

  In step S30, the scanning interval of the detection window region is obtained using the reliability obtained in the determination in the previous detection window region. As described above, the reliability is the number assigned to the last strong classifier among the strong classifiers that output the discrimination result that the detection target exists (T in the figure) in the cascade type classifier 320 (first stage). The number of stages from This process is performed by the detection window control means 32.

  Specifically, the reliability determined in the previous detection window area stored in the determination information history 40 is read, and the corresponding scanning interval is selected from a correspondence table prepared in advance. FIG. 4 shows an example of the correspondence table 400.

  The lower diagram of FIG. 2 shows, as a graph 210, how the reliability determined when the detection window region is shifted pixel by pixel in the X direction at Y = y1 of the input image 200 changes. As the area where the person is reflected is approached, the identification number (reliability) for outputting the identification result that the detection target is present in the cascade discriminator 320 increases, and the identification is performed as the person moves away from the area where the person is reflected. The number (reliability) decreases. From this, it can be said that the image of the detection window area does not resemble a person when the reliability is small, and resembles a person when the reliability is large, and the correspondence table 400 scans as the reliability decreases. The interval is set to be large, and the scanning interval is set to be smaller as the reliability is higher. The scanning interval is larger than 0 (pixel) and smaller than the width or height of the detection window region.

  For example, when the input image has a width of 320 pixels and a height of 240 pixels, the detection window area has a width of 64 pixels and a height of 128 pixels, and the number of stages of the strong classifiers of the cascade classifier 320 is 13, V1 = 8, V2 = 8, V3 = 7, V4 = 7, V5 = 6, V6 = 6, V7 = 6, V8 = 5, V9 = 5, V10 = 5, V11 = 4, V12 = 4, V13 = 4.

  Note that immediately after the size of the detection window area is changed in step S20, that is, at the start of scanning of the image in the new detection window area, the scanning interval is set to a preset initial value.

  In step S40, the position of the detection window area is determined. This process is performed by the detection window control means 32. A method for determining the position of the detection window region will be described with reference to the flowchart of FIG. The upper left coordinate of the detection window area is called a start point, the start point of the detection window area to be determined is (SX, SY), and the previous start point of the detection window area is (BX, BY).

  In step S401, it is determined whether or not the size of the detection window area has been changed. If the size of the detection window area has just been changed, the start point (SX, SY) of the detection window area is set to (0, 0) in step S405, and the process proceeds to step S50. If it is not the first process after the size of the detection window area is changed, the process proceeds to step S402.

  In step S402, the start point (BX, BY) of the previous detection window area is read. In step S403, it is determined whether scanning has been performed from the start point (BX, BY) of the previous detection window area read in step S402 to the right end of the image. When scanning to the right edge of the image is completed in the detection window area, that is, when the right edge BX + W of the detection window area (W is the width of the detection window area) coincides with the right edge of the image, the detection window area in step S406 Is set to (SX, SY) = (0, BY + q), and the process proceeds to step S50. However, q is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, q = 4 pixels is set. If not scanned to the right end, the process proceeds to step S404.

  In step S404, the detection window area is shifted in the X direction by the scanning interval determined in step S30, and the start point of the detection window area is determined (SX, SY) so as not to be shifted in the Y direction. That is, the detection window region start point (SX, SY) = (BX + scanning interval determined in step S30, BY). Thereafter, the process proceeds to step S50.

  In step S50, a similarity indicating how much the image in the detection window area set in step S40 resembles a person is obtained. This is a discrimination process of the cascade discriminator 320 shown in FIG. The identification process will be described with reference to the flowchart of FIG.

  In step S501, the HOG feature amount is calculated from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  In step S502, it is determined whether or not all strong classifiers have been investigated. If all strong classifiers have been investigated, the process proceeds to step S505. If not completed, the process proceeds to step S503, and the next strong classifier is investigated. In step S503, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S504, it is determined whether the similarity calculated in step S503 is greater than a preset threshold value. If it is larger than the threshold value, the process proceeds to step S502, and if it is less than the threshold value, the process proceeds to step S505. In step S505, the size (width and height) of the detection window region, the center coordinates, the similarity, and the reliability are associated and recorded as the determination information history 40, and the process proceeds to step S60.

  In step S60, it is determined whether or not the detection process has been completed for detection window regions of all sizes. If the inspection is completed for all detection window regions, the process proceeds to step S70, and if not, the process returns to step S20.

  In step S70, the position where the person is finally captured from the size (width and height), center coordinates, similarity, and reliability of each detection window area stored in the determination information history 40 obtained in step S50. To decide. This process is performed by the judging means 36.

  The processing of steps S701 to S702 is performed for all detection window regions stored as the determination information history 40 obtained in step S50. In step S701, it is determined whether or not the reliability matches the identification number of the last stage of the strong classifier included in the cascade classifier 320, and the similarity is greater than a threshold value. If the condition is satisfied, step S702 is performed. If not, the next detection window region is determined. In step S702, the detection window area that satisfies the condition in step S701 is added to the human candidate area.

  As shown in FIG. 2, candidate candidate areas obtained by repeating steps S701 to S702 on the information stored in the determination information history 40 are areas 203 and 204 near the area where the person is shown. May be extracted multiple times. In steps S703 to S705, a human region is finally selected from a plurality of human candidate regions.

  In step S703, using the size and center coordinates of the detection window area extracted as the human candidate area, the human candidate areas that overlap a certain area (for example, more than half the area of the detection window area) are grouped together. . In step S704, the detection window region having the highest similarity among the human candidate regions is selected as the human region for each group created in step S703. In step S705, the detection window area selected in step S704 is set as a human area, and the process proceeds to step S80.

  In step S80, if there is at least one human area based on the human area determined in step S70, the detection window information of the determined human area is output as an abnormal signal together with the image. This processing is performed by the output unit 5.

  In the present embodiment, the scanning interval in the X direction is changed according to the reliability determined by the determination in each discriminator of the cascade type discriminator 320. Similarly, the scanning interval in the Y direction is changed. It may be changed. For example, q at the time of determining the start point (SX, SY) of the next scan line in accordance with the average of the reliability outputs from the cascade discriminator 320 in each detection window region one scan line before. The value of can be changed.

<Second Embodiment>
In the second embodiment, only the configuration of the discriminator 34 in the first embodiment is different. As shown in FIG. 9, the discriminator 34 in the present embodiment determines the human area based on the number of strong discriminators connected in parallel. That is, in the first embodiment, the processing is terminated when it is determined in the cascade classifier 320 that any strong classifier is not a person, but in this embodiment, all strong classifiers are used. When the number of strong classifiers that have been identified and that the search window area has determined to include people is greater than the number of strong classifiers that have been determined to be otherwise, the number of classifiers that have been determined to include people It outputs to the determination means 36.

<Third Embodiment>
In the third embodiment, only the configuration of the discriminator 34 in the first embodiment is different. The discriminator 34 in the present embodiment stores feature information including parameters such as shape, brightness, and edge information of a person on the image in advance in the storage unit 4 as a template, and the image and feature information of the detection window region. And the similarity is output to the determination unit 36 as the reliability of the humanity of the detection window region.

  As described above, according to each of the above-described embodiments, it is possible to reduce the processing burden in detecting an object from an image and eliminate detection omission.

  Moreover, although this Embodiment demonstrated the aspect which implement | achieves the function of each part of the object detection apparatus 1 with one computer, it is not limited to this. The functions of each part of the object detection device 1 can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by a single computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

It is a functional block diagram which shows the structure of the object detection apparatus in embodiment of this invention. It is a figure explaining the detection process of an object. It is a figure which shows the structure of the discriminator in embodiment of this invention. It is a figure which shows the example of the conversion table in embodiment of this invention. It is a flowchart which shows the process in the object detection apparatus in embodiment of this invention. It is a flowchart which shows the position setting process of the detection window area | region in the object detection apparatus in embodiment of this invention. It is a flowchart which shows the identification process in the object detection apparatus in embodiment of this invention. It is a flowchart which shows the determination process in the object detection apparatus in embodiment of this invention. It is a figure which shows the structure of the discriminator in embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Image acquisition part, 3 Signal processing part, 4 Storage part, 5 Output part, 30 Detection window area | region selection means, 32 Detection window control means, 34 Classifier, 36 Determination means, 40 Determination information log | history, 200 Input image, 201, 202 detection window area, 203, 204 area, 310 detection window extraction means, 320 cascade type discriminator, 321, 322, 323 strong discriminator, 330 similarity / reliability management means, 400 correspondence table.

Claims (4)

An object detection device for detecting a detection target from an image,
An image acquisition unit for acquiring the image;
A detection window region image in which a detection window is set is captured from the image input from the image acquisition unit, and it is determined whether or not the detection target is present in the detection window region image, and in the determination A discriminator that outputs a reliability indicating the degree of certainty of the presence of the detection object;
Detection window control means for setting the scanning interval of the subsequent detection window to be shorter as the reliability is higher;
Determining means for determining whether or not the detection object exists based on an output from the discriminator;
An object detection apparatus comprising:   The object detection apparatus according to claim 1, wherein the detection window control unit sets the scanning interval to a range that does not exceed a size of the detection window. The discriminator outputs a similarity indicating the degree of similarity between the detection window region image and the detection target;
The determination means determines that the detection window region image having a high similarity from the detection window region images having a reliability equal to or higher than a predetermined value and overlapping each other is a detection target image including an image of the detection target. The object detection apparatus according to claim 1, wherein: The discriminator includes a cascade type discriminator in which a plurality of strong discriminators are connected in series, and each of the plurality of strong discriminators connected in series has the detection target in the detection window region image. Output the identification result of whether or not
The number of stages from the first stage of the strong classifier of the last stage among the strong classifiers that output the identification result that the detection target exists in the cascade classifier is defined as the reliability. The object detection apparatus according to one.

Images