JP2011210054A - Object detection device and learning device for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem of an object detection device detecting an object from a voting result about a plurality of feature points appearing in an image of the object, wherein high-accuracy detection is difficult because a difference between votes caused by the likelihood of variation of the feature points occurs.SOLUTION: A detection storage part 12 stores information about a feature point distribution in an object image and a relative position of a reference point from a representative position of the feature point distribution as feature point information 120. A vote part 141 obtains, in the input image, a voting value of a feature point in each position of the input image according to the feature point distribution moved such that the representative position accords with the relative reference point provided in the relative position of a detection position of the feature point. An object decision part 142 determines a position of the reference point inside the input image based on a point wherein a total value of voting values of the respective feature points in the respective positions of the input image exceeds a prescribed detection threshold value and becomes locally maximal, and determines that the object image is present in the position.

Description

  The present invention relates to an object detection apparatus that detects an object appearing in an input image, and a learning apparatus used for the learning.

  In recent years, active research has been conducted to detect the presence of people, faces, and the like from images from surveillance cameras and digital still cameras. In the detection process, a search method using a pattern matching device or a classifier is used. That is, windows are set at various locations in the image, the window images are input to a pattern matching device and a discriminator, and the detection results output by these are totaled to detect an object at a position where the total value is high.

  The target object in the image is not necessarily an image of the entire target object, and a part of the target object may be hidden by another object. In order to detect an object that is partially concealed, conventionally, a plurality of feature points of the object are detected, and the detection results of these feature points are integrally determined.

  For example, in the prior art described in Patent Document 1, a plurality of feature points and the position (relative position) of the reference point of the sample image with respect to the position of the feature point are learned in advance for the sample image of the object and input. Voting is performed on the relative position viewed from the feature point detected from the image, and when the total value of the voting exceeds a threshold value in the input image, it is determined that the object exists. In other words, if multiple feature points are detected in the same positional relationship as in the sample image, votes from those feature points gather in one place in the input image, and the target value is detected when the total number of votes exceeds the threshold value. It is done.

JP-A-9-21610

  The feature points include those that tend to vary in relative position with respect to the reference point and those that do not. For example, if the object is a person, the head has a small range of motion, so the variation in the relative positions of the feature points around the head and the reference point is relatively small, but the leg has a large range of motion, so the legs are around the legs. The relative position between the feature point and the reference point varies greatly.

  This means that votes relating to feature points with small variations tend to gather in one place, but votes relating to feature points having large variations are difficult to gather in one place. Therefore, if these feature points are uniformly voted, there is a problem that detection omission is likely to occur.

  The present invention has been made to solve the above-described problem, and an object detection apparatus capable of correcting an object difference with a high degree of accuracy by correcting a disparity in voting due to the variability of the relative positions of feature points and reference points. It is another object of the present invention to provide a learning device used for constructing the object detection device.

  An object detection device according to the present invention detects an object appearing in an input image, and has a plurality of feature points each having an image feature indicating a characteristic of a target object image obtained by photographing the object set in advance. In addition, a storage unit storing feature point information including a relative position between a predetermined reference point and a feature point in the target object image and a variation degree of the relative position, and the feature point is detected from the input image The feature point detection unit and the feature point detected by the feature point detection unit obtain a vote value with a distance characteristic according to the degree of variation around a relative position with respect to the position of the feature point in the input image. A voting unit for calculating, and an object determination unit for counting the voting values at each position in the input image to determine the presence of the object.

  In another object detection apparatus according to the present invention, the distance characteristic of the voting unit increases the distance range of the voting position from the center as the variation degree increases.

  In still another object detection apparatus according to the present invention, the distance characteristic of the voting unit gradually attenuates the voting value according to a distance away from the center as the degree of variation increases.

  A learning device according to the present invention generates the feature point information used in the object detection device, and includes a sample image storage unit in which a plurality of sample images taken of the object are stored; A sample feature point extraction unit that extracts a sample feature point having a predetermined image feature from each sample image, and a clustering unit that generates a cluster composed of the sample feature points having similar positions and image features between the sample images And for each cluster, information on the sample distribution regarding the distribution of the position of the sample feature point is obtained by statistical analysis, and further, a relative position of a predetermined reference point in the sample image from a predetermined representative position in the sample distribution The feature point is obtained by obtaining the sample relative position and the sample distribution information and the sample relative position for each cluster as the feature point distribution information and the relative position. Has a characteristic point information generation unit for generating a multi-address, the.

  According to the object detection device according to the present invention, the disparity of voting due to the variability of feature points is corrected, and the object can be detected with high accuracy. According to the learning device according to the present invention, An object detection device can be constructed.

It is a block diagram which shows the structure of the outline of the target object detection apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows an example of a target object sample image. It is the schematic diagram which expressed typically a part of parameter of feature point information about the example of a feature point on the two-dimensional area | region corresponding to a target object sample image. It is the schematic diagram which represented the parameter group which comprises feature point information in tabular form. It is a schematic diagram which shows the example of the feature point detected in the input image. It is a schematic diagram which illustrates the mode of voting about the some feature point detected by the same target object. It is a figure for demonstrating the mode of a target object determination process. It is a flowchart which shows the operation | movement of the outline of the target object detection apparatus which concerns on embodiment of this invention. It is a general | schematic flowchart of a feature point detection process and a voting process. It is a general | schematic flowchart of a target object determination process. It is a block diagram which shows the schematic structure of the learning apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the relationship between a sample point and a cluster. It is a flowchart which shows the operation | movement of the outline of the learning apparatus which concerns on embodiment of this invention.

  Hereinafter, an object detection device 1 and a learning device 2 which are embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings. For example, the object detection apparatus 1 uses a monitoring image obtained from the monitoring space as an input image, and detects an object appearing in the input image. In this embodiment, a person is an object, and an intruder is detected by detecting a human feature point in a monitoring image obtained from the monitoring space, and an abnormal signal is output when the intruder is detected. The learning device 2 generates feature point information used for the object detection device 1 by learning.

[Object detection device]
FIG. 1 is a block diagram illustrating a schematic configuration of an object detection device 1 according to the embodiment. The object detection device 1 includes an imaging unit 10, an image acquisition unit 11, a detection storage unit 12, a feature point information setting unit 13, a detection control unit 14, and a detection output unit 15. The image acquisition unit 11 is connected to the imaging unit 10, and the image acquisition unit 11, the detection storage unit 12, the feature point information setting unit 13, and the detection output unit 15 are connected to the detection control unit 14.

  The imaging unit 10 is a surveillance camera and is installed in a surveillance space. For example, the monitoring camera is installed on the ceiling of the monitoring space over the monitoring space. The monitoring camera images the monitoring space at a predetermined time interval (for example, 1 second), and sequentially outputs monitoring images in which each pixel is expressed by a multi-gradation pixel value.

  The image acquisition unit 11 is an interface circuit that acquires a monitoring image captured by the imaging unit 10 and imports the monitoring image into the detection control unit 14. Hereinafter, an image input from the image acquisition unit 11 to the detection control unit 14 is referred to as an input image.

  The detection storage unit 12 is a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk, and stores programs and data used by the detection control unit 14. The detection storage unit 12 inputs and outputs these programs and data to and from the detection control unit 14. The data stored in the detection storage unit 12 includes feature point information 120 and a voting image 121.

  The feature point information 120 includes a feature point number for identifying a feature point for each of a plurality of feature points having an image feature of the target object, a relative position between a reference point position such as a barycentric position in the target object image, and a feature point position. It consists of detection criteria for detecting feature points from the image and the degree of variation in relative position.

  The feature point information 120 is generated in advance by the learning device 2 to be described later based on a large number of object specimen images obtained by imaging the object.

  FIG. 2 is a schematic diagram illustrating an example of an object specimen image. In the present embodiment, the detection target is a person, and the target specimen image is a whole person image. Each object specimen image is normalized to a vertically long rectangle of 64 pixels in the width (horizontal) direction × 128 pixels in the height (vertical) direction according to the shape of the person, and the barycentric coordinates (32, 64) are used as the object reference. Set as point B.

FIG. 3 is a schematic diagram schematically showing some parameters of the feature point information 120 on a two-dimensional region corresponding to the object specimen image with respect to an example of feature points. ) Correspond to different feature points. FIG. 4 is a schematic diagram showing the parameter group constituting the feature point information 120 in a table format. Each parameter constituting the feature point information 120 will be described with reference to FIGS. Here, the number of set feature points is M (> 1), and 1 to M feature point numbers (feature points #) are assigned to each feature point. The positions of the M feature points (“x” in FIG. 3) are different from each other, and for each of the feature points # 1 to #M, vectors R _{1 to} R _M from the position to the object reference point B are the features. It is stored in the feature point information 120 as the relative position of the point. Further, the feature point information 120 stores N-dimensional vectors A _{1 to} A _M representing the feature quantities of the object specimen images at the respective positions of the feature points # 1 to #M as detection criteria for the M feature points. Has been.

  Features are known Shape Contexts, Histograms of Oriented Gradients (HOG), Navneet Dalal and Bill Triggs, “Histograms of Oriented Gradients for Human Detection”, In Proceedings of IEEE Conference Computer Vision and Pattern Recognition 2005).

  The shape context is a feature quantity representing the distribution characteristics of edges around the feature point, and the data is in a vector format. The shape context is calculated by analyzing the image in the analysis window set around the feature point, and the index of each element of the vector is the difference between the small area divided into the analysis window and the quantized edge direction. Corresponding to the combination, the value of each element corresponds to the sum of the strengths of the edges having the edge direction represented by the index in the small area represented by the index. HOG is also a vector quantity representing the distribution characteristic of the luminance differential value around the feature point. Both the shape context and the HOG represent the distribution characteristics of the luminance gradient around the feature points, and are suitable for detecting an object because they are robust against illumination fluctuations.

Variation degree of the M feature points is also a feature quantity that may vary for each feature point, the variation of V ₁ ~V _M for each of the feature points #. 1 to # M are stored as feature information 120. In the present embodiment, the degree of variation is a variance value in a plurality of learned sample images related to the relative position between the feature point position and the reference point position. The degree of variation has a component vx in the x direction (horizontal direction) and a component vy in the y direction (vertical direction). In FIG. 3, a solid oval centered on the feature point represents a range corresponding to a confidence interval of a predetermined reliability, and is obtained from the variance value. In addition, the display of the square root (that is, standard deviation) of each of vx and vy described corresponding to the radii of the ellipse in the x direction and y direction is the ratio of the radii in the x direction and the standard deviation in the y direction. It shows that it is given.

  The distribution of the relative position between the feature point position and the reference point position may be analyzed to obtain a distribution main axis, which may be represented by two axes, an axis orthogonal to the main axis.

Variation in the relative position between the feature point position and the reference point position is caused by variations in posture, individual differences in proportion, and the like. A feature point # 1 shown in FIG. 3A represents a feature point near the head, and a feature point #M shown in FIG. 3C represents a feature point near the leg. Since the range of motion of the leg is larger than that of the head, the degree of variation of the feature point #M is larger than the degree of variation of the feature point # 1 (vx ₁ > vx _M , vy ₁ > by _M ).

  The voting image 121 is information used for object determination, and relative to a plurality of feature points detected from the input image on a frame having the same size as the input image in consideration of the degree of variation of each feature point. It is an image which shows the result of having voted for the position. In the present embodiment, as described later, the voting image 121 is prepared for each detection magnification in response to the object detection processing being performed at a plurality of magnifications (detection magnifications).

  The feature point information setting unit 13 inputs the feature point information 120 from the outside, such as a USB terminal, a CD drive, a network adapter and other interface circuits and their respective drivers and programs, and the input feature point information 120 in the detection storage unit 12. Consists of programs to be stored. The feature point information 212 generated by the learning device 2 is input via the feature point information setting unit 13 and stored as feature point information 120 in the detection storage unit 12.

  The detection control unit 14 is configured using an arithmetic device such as a DSP (Digital Signal Processor) or an MCU (Micro Control Unit). The detection control unit 14 processes the input image from the image acquisition unit 11 to determine the presence or absence of a person, and performs processing to output an abnormal signal to the detection output unit 15 when a person is detected. Specifically, the detection control unit 14 reads out and executes a program from the detection storage unit 12, and functions as a feature point detection unit 140, a voting unit 141, an object determination unit 142, and an abnormality determination unit 143, which will be described later.

  The feature point detection unit 140 detects each feature point from the input image, detects the feature point number of the detected feature point, the position (detection position) in the input image where the feature point is detected, and the detection of the feature point The feature point detection information in which the degree and the detection magnification when the feature point is detected is associated is output to the voting unit 141.

  The feature point detection unit 140 sets an analysis window around each position in the input image, extracts a feature amount in the analysis window, compares the feature amount with a detection criterion for each feature point, and calculates a degree of detection. . If the calculated degree of detection exceeds a preset feature point detection threshold Tp, the feature point is detected at the position. If the degree of detection does not exceed Tp, the feature point is not detected at the position. . The feature amount is the shape context described above in the present embodiment, but may be a HOG. Note that the feature amount needs to be the same type as the feature amount used when the detection criterion is learned by the learning device 2 described later.

  In this embodiment, feature quantities are stored as detection criteria. In this case, the feature point detection unit 140 performs pattern matching to detect feature points. That is, the Euclidean distance d between the extracted feature quantity and the detection reference feature quantity is calculated as a degree of detection. If d is equal to or less than Tp, a feature point is detected.

  In another embodiment in which the discriminant function is stored as a detection criterion, the feature point detector 140 inputs the extracted feature quantity into the discriminant function, calculates the likelihood that is the output value as the detectability, and the likelihood. If is larger than Tp, a feature point is detected. That is, the feature point detection unit 140 is configured to operate as a discriminator.

Corresponding to the fact that the size of the object captured in the input image varies, the feature point detection unit 140 adjusts the detection magnification to match the variety of the object size when detecting the feature point. Process. Here, the detection magnification α is a magnification of the size of the object captured in the input image when the size of the object captured in the object specimen image is used as a reference. Specifically, in order to match the size of the object captured in the input image with the size of the object specimen image, the input image is enlarged or reduced in accordance with preset multiple detection magnifications. By the enlargement / reduction, the input image becomes 1 / α of the original size. The detection magnification α is, for example, (1.05) ³ times, (1.05) ² times, 1.05 times, 1.0 times, 1 / 1.05 times, 1 / (1.05) ² times, 1 /(1.05) Set to 7 levels, ³ times. Enlarging / reducing processing can be performed by a known bilinear interpolation method or the like.

The voting unit 141 obtains a voting value for each feature point detected in the input image. Specifically, the voting unit 141 refers to the feature point information 120 and the feature point detection information from the feature point detection unit 140, and for each feature point detected by the feature point detection unit 140, A relative reference point Q shifted from the detection position P by the relative position R of the feature point is calculated, and a voting value of a distance characteristic is set according to the variation degree of the feature point with the Q as a center. Then, the relationship between the set voting position and the voting value is output to the object determining unit 142. When the detection magnification at the time of detection is α, the relative reference point Q of the feature point #j is calculated by the following equation.
Q = (P + R _j ) / α (1)

  Here, when the detected feature point is true, the relative reference point Q represents the position where the object reference point B exists. In other words, the relative reference points Q of the feature points detected from the same object ideally match each other. Actually, the detected feature point has a deviation from the representative position of the feature point as the starting point of the vector R representing the relative position due to the variation, and the feature points detected from the same object are correspondingly shifted. A distance also occurs between the relative reference points Q.

When the detection position P of the feature point #j is represented as (p _x , p _y ) and the relative reference point Q is represented as (q _x , q _y ), the vote value f (x, y) of each pixel (x, y) in the input image ) Can be defined by the following equation:

F (x, y) expressed by the equation (2) is a two-dimensional normal distribution in which the average is the relative reference point Q and the variance is the variation degree V _j . The vote value defined by the equation (2) is given a distance attenuation characteristic that shows a maximum value at the relative reference point Q and decreases as the distance from the relative reference point Q increases. A slower distance attenuation characteristic is given as the variation degree V _j is larger, and a steeper distance attenuation characteristic is given as the variation degree V _j is smaller. That is, the voting unit 141 decreases the voting value according to the distance from the relative reference point. That is, the closer the relative reference point is to the object reference point B, the higher the voting value is set at the object reference point B.

  w is a function of the degree of detection, and is set so as to increase as the degree of detection exceeds the feature point detection threshold Tp. That is, the vote value f (x, y) is weighted more as the degree of detection exceeds the feature point detection threshold Tp. That is, a higher voting value is set for a feature point with higher detection reliability. For example, w = exp (−kd) is set for the Euclidean distance d. Here, k is a preset positive constant. Further, for example, in another embodiment in which the detection degree is the likelihood L, w = L can be set.

The vote value function f can be defined as other than the expression (2). For example, a quadrangular pyramid function expressed by the following expression (3) or a conical function expressed by the expression (4) It can be specified by.

According to the functions of the expressions (3) and (4), in addition to the distance attenuation characteristics as described above, the distance range in which the voting value is set is limited to a width corresponding to the degree of variation. Also in the voting function of the formula (2), for example, it can be limited to a distance range that satisfies the following formula (5). Incidentally, equation (5) represents a 3σ confidence interval in a normal distribution. By limiting in this way, the voting value can be set only in a range where it is certain that the true object reference point exists, and the reliability of object detection is improved.

  As still another embodiment, a configuration may be adopted in which a voting value of f (x, y) = w is set in the range of equation (5).

  FIG. 5 is a schematic diagram illustrating an example of feature points detected in the input image 400. FIG. 5 shows the position of the feature point where the “x” mark is detected. For example, feature points 401, 402, and 403 are detected on the shoulder, left foot, and right foot of the same person.

  FIG. 6 is a schematic diagram illustrating a state of voting for the feature points 401 to 403 detected on the same object. It is assumed that the true value of the center of gravity of the target object captured in the input image 400 is (x0, y0). An image 430 shown in FIG. 6 is a part of the input image 400. A graph 450 shows a state of voting according to the present embodiment at a position along the straight line y = y0 in the input image 430, where the horizontal axis represents the position and the vertical axis represents the vote value f (x, y). . Regions 411 to 413 represent voting ranges set around the relative reference points 421 to 423 for the feature points 401 to 403, respectively. Graphs 451 to 453 are voting values in the voting range, and represent voting values for the feature points 401 to 403, respectively.

  The difference in position between the center of gravity (x0, y0), which is the object reference point B, and the relative reference points 421 to 423 of each feature point corresponds to a shift in the detection position of each feature point. In (x0, y0), not only the vote values of feature points 401 and 402 detected with a relatively small shift in the x direction but also the vote values of feature points 403 detected with a relatively large shift are set. .

  On the other hand, for comparison with the graph 450 of the present embodiment, the graph 470 shows the state of voting when the degree of variation V is a constant value common to all feature points. Graphs 471 to 473 in graph 470 correspond to graphs 451 to 453 in graph 450, respectively. The graph 470 is fundamentally different from the graph 450 in that the vote value (graph 473) of the feature point 403 having a large deviation is not set to (x0, y0).

  A large feature point variation means that the feature point may be detected in a wide range. Conversely, a small feature point variation means that the feature point is detected in a narrow range. Means that. As described above, the object detection device 1 sets a voting value of the distance attenuation characteristic according to the degree of variation of the feature points, or sets a voting value within a distance range according to the degree of variation of the feature points. Effective voting is performed on target reference points where the target is truly present even from feature points that are likely to vary, thus avoiding voting biased to feature points that are less likely to vary, thereby preventing false detection and detection omission of the target. Is possible.

  In addition, since voting according to the existence probability of feature points is performed by giving a distance attenuation characteristic to the voting value, it is possible to perform voting with high accuracy while preventing unreasonably high voting from feature points that tend to vary.

  As described above, the voting unit 141 sets the voting value f (x, y) for each position (x, y) for each feature point. The object determination unit 142 accumulates and adds the vote values set at the respective positions to the pixel values at the positions in the vote image 121 (primary aggregation). The setting of the vote value for each feature point is performed for each detection magnification, and addition to the vote image is also performed for each detection magnification using a vote image corresponding to the detection magnification.

  The object determination unit 142 further performs a counting process (secondary counting) on the voting value at each position of the input image based on the voting image for which the voting value is set by the voting unit 141. Specifically, the voting value of the same object may be set across a plurality of detection magnifications due to individual differences in the imaging state of the object and the proportion. Therefore, the object determination unit 142 further adds the vote values at the same position where the detection magnifications are adjacent (secondary aggregation). As a result, the size balance between the parts due to the imaging state and individual differences can be absorbed, and the detection omission of the object can be prevented. Then, it is determined that there is an object at a position where the aggregate value exceeds a preset object detection threshold To, and on the other hand, if there is no position where the aggregate value exceeds To, no object exists in the input image. And the determination result is output. The determination result is input to the abnormality determination unit 143.

  In addition, the total value may exceed To at a plurality of positions in the vicinity of the position where the object truly exists. Therefore, the object determination unit 142 divides the voting image 121 into a plurality of blocks, detects a position (peak point) where the total value is maximum for each block, and uses only the total value of the peak points as the object detection threshold To. Compare. The block size is set smaller than the size of the object on the input image enlarged or reduced according to the detection magnification. This can prevent false detection of the object.

  The target object determination unit 142 generates target object detection information that associates the position in the input image determined to have the target object as a determination result, the total value at the position, and the detection magnification at which the total value is calculated. .

  FIG. 7 is a diagram for explaining the state of the object determination process. FIG. 7A shows the input image 500 and the positions (x1, y1) and (x2, y2) where the object is imaged. It is a schematic diagram shown. FIG. 7B is a schematic diagram showing a state in which voting values for feature points detected from the input image 500 shown in FIG. 7A are set for voting images 510 to 516 having different detection magnifications. It is. FIG. 7B shows an xyα three-dimensional space in which the sizes of the voting images 510 to 516 are aligned in the x direction and the y direction, and the voting images are arranged in the order orthogonal to the x axis and the y axis in the order of the detection magnification α. Represents. In FIG. 7B, the circle represents the voting range, and it can be seen that the voting is concentrated at the positions (x1, y1) and (x2, y2) where the object is imaged. When these are totaled, peaks exceeding the object detection threshold value To are detected at the positions (x1, y1) and (x2, y2), and the presence of the object is determined at the positions (x1, y1) and (x2, y2). .

  When the object determination unit 142 determines the presence of the object, the abnormality determination unit 143 outputs an intrusion abnormality signal to the detection output unit 15 assuming that an intrusion abnormality is detected.

  The detection output unit 15 is an interface circuit that is connected to an external device and outputs an intrusion abnormality signal to the external device. The external device is an alarm display means such as a speaker, a buzzer, or a lamp for alarming the presence of an intruder, a remote center device connected via a communication network, and the like.

  Next, operation | movement of the target object detection apparatus 1 is demonstrated. FIG. 8 is a flowchart showing a schematic operation of the object detection apparatus 1. For example, when the device administrator turns on the power, each unit starts operating. The image acquisition unit 11 inputs images captured at predetermined time intervals to the detection control unit 14. The detection control unit 14 repeats the process consisting of steps S10 to S18 each time an image is input.

  When an image is input (S10), the feature point detection unit 140 of the detection control unit 14 detects a feature point from the input image, and the voting unit 141 of the detection control unit 14 adds a vote according to the detection result to the vote image 121. This is performed (S12).

  FIG. 9 is a schematic flowchart of the feature point detection process and the voting process (S12). The feature point detection process and the voting process will be described with reference to FIG.

  The feature point detection unit 140 sequentially sets the seven detection magnifications to the attention magnification (S120), and executes a loop process that repeats the processes of steps S121 to S132 for all the detection magnifications.

  In the detection magnification loop processing, first, when the attention magnification is other than 1, the feature point detection unit 140 generates an input image having a size corresponding to the attention magnification by performing enlargement or reduction (S121). The feature point detection unit 140 sequentially sets all the pixel positions of the input image at the center of the analysis window, and extracts the feature amount in the analysis window at each set position (S122). The extracted feature amount is associated with the extraction position and temporarily stored in the detection storage unit 12 as feature amount information. By calculating and storing the feature amount at this stage and making it available at any time in later processing, it is possible to eliminate unnecessary duplication calculation. In addition, the voting unit 141 initializes each pixel value of the voting image 121 with the attention magnification to 0 (S123).

  Next, the feature point detection unit 140 sequentially sets M feature points #m (1 ≦ m ≦ M) stored in the feature point information 120 as feature points of interest in the loop processing of the detection magnification. (S124) Further, each pixel position in the input image is sequentially set as a target position (S125), and a loop process for repeating the processes in steps S126 to S131 is executed for all combinations of feature points and pixel positions.

  In the loop processing related to the feature point and the pixel position, the feature point detection unit 140 reads the reference point of the target feature point from the feature point information 120, and further calculates the feature amount of the target position from the feature amount information generated in step S122. The degree of detection is calculated by comparing the feature amount of the target position with the detection criterion of the target feature point (S126), and the calculated degree of detection is compared with the feature point detection threshold Tp (S127).

  If the degree of detection exceeds the feature point detection threshold Tp (“YES” in S127), it is determined that the feature point of interest is detected at the target position, and the feature point detection unit 140 applies the attention magnification and feature point of interest to the voting unit 141. The feature point number, the target position, and the detection degree are notified. The voting unit 141 reads the degree of variation V and the relative position R of the feature point of interest from the feature point information 120, and notifies the noticed magnification α, the position of interest P and the degree of detection d, and the read degree of variation V and the relative position R. Is substituted into the expressions (1) and (2) to calculate the vote value f (x, y) for each pixel position (x, y) in the input image (S128), and each calculated pixel position The voting value f (x, y) of (x, y) is added to the pixel value of the corresponding pixel position (x, y) in the voting image 121 of the attention magnification (S129). The addition processing in step S129 corresponds to primary aggregation. On the other hand, when the degree of detection is equal to or less than Tp (“NO” in S127), steps S128 and S129 are omitted because the feature point of interest has not been detected at the target position.

  When the entire input image has been scanned for all feature points and magnifications (“YES” in S130, “YES” in S131, and “YES” in S132), the feature point detection process and the voting process are completed). To do.

  When the feature point detection process and the voting process are completed, the process of the object detection device 1 proceeds to the object determination process S14 as shown in FIG. In the object determination process S14, as described below, the object determination unit 142 of the detection control unit 14 determines whether or not an object exists in the input image based on the vote image created in step S12. Is determined.

  FIG. 10 is a schematic flowchart of the object determination process (S14). The object determination process (S14) will be described with reference to FIG.

  For each detection magnification, the target object determination unit 142 adds the corresponding pixel values of the voting images with adjacent detection magnifications, that is, the primary aggregation values (S140), and divides the voting image of each detection magnification into blocks. Peak pixels are detected for each block (S141). The addition processing in step S140 corresponds to secondary aggregation.

  Next, the object determination unit 142 sequentially sets each peak pixel as a target peak pixel (S142), and executes a loop process that repeats the processes of steps S143 to S145 for all peak pixels.

  In the loop processing related to the peak pixel, the object determination unit 142 compares the pixel value (secondary aggregate value) of the peak pixel of interest with the object detection threshold value To (S143). If the target value is detected at the position of the target peak pixel if the total value is larger than To (“YES” in S143), the position of the target peak pixel, the pixel value of the target peak pixel, and the target peak pixel belong to it. Object detection information in association with the detection magnification of the voting image is generated and stored in the detection storage unit 12 (S144). On the other hand, when the total value is equal to or less than To (“NO” in S143), step S144 is omitted.

  When all the peak pixels have been processed in this way (“YES” in S145), the object determination process S14 ends.

  When the object determination unit 142 finishes the process, the abnormality determination unit 143 of the detection control unit 14 refers to the detection storage unit 12 to confirm the presence / absence of the object detection information (S16), and even one object detection information exists. If it is stored, the object is detected ("YES" in S16), an intrusion abnormality signal is output to the detection output unit 15, and an alarm is output to the detection output unit 15 (S18).

  When the above process is completed, the process returns to step S10 again.

  In the above embodiment, the image acquisition unit 11 is connected to the imaging unit 10, and the detection control unit 14 detects the object by online processing. However, the image acquisition unit 11 may be connected to the recording device, and the detection control unit 14 may detect an object by offline processing.

  In the above-described embodiment, the feature point detection unit 140 scans all pixel positions in the input image. However, the feature point detection unit 140 preliminarily detects blobs and corners from the input image, and scans only the pre-detected position and its periphery. You may do it. At this time, if the sample point setting unit 220 detects a blob, the feature point detection unit 140 also preliminarily detects the blob, and if the sample point setting unit 220 detects a corner, the feature point detection unit 140 also reserves the corner. To detect.

  In the above embodiment, the degree of variation calculated by the learning device 2 is input from the feature point information setting unit 13 as part of the feature point information 212 and stored in the detection storage unit 12. As another embodiment, the feature point information setting unit 13 further includes an operation input device such as a keyboard and a mouse, and the administrator of the object detection device 1 operates the feature point information setting unit 13 to input the degree of variation. It is good.

[Learning device]
FIG. 11 is a block diagram illustrating a schematic configuration of the learning device 2 according to the embodiment. The learning device 2 includes a learning operation unit 20, a learning storage unit 21, a learning control unit 22, and a learning output unit 23. The learning operation unit 20, the learning storage unit 21, and the learning output unit 23 are connected to the learning control unit 22.

  The learning operation unit 20 is a user interface device such as a keyboard and a mouse, and is operated by an administrator of the device to give a learning start instruction and an instruction to output feature point information to the learning control unit 22.

  The learning storage unit 21 is a storage device such as a ROM, a RAM, and a hard disk, and stores programs and data used by the learning control unit 22. The learning storage unit 21 inputs and outputs these programs and data to and from the learning control unit 22. The data stored in the learning storage unit 21 includes a sample image 210, sample point information 211, and feature point information 212.

  The sample image 210 is an image serving as a basis for creating the feature point information 212, and is stored in advance prior to the learning. The sample image 210 includes a large number (several thousands to several tens of thousands) of target object images in which the target is imaged, and a large number (several thousands to tens of thousands) of the non-target samples in which the target is not captured. It consists of an image. Each of the sample images 210 is given a sample number for identifying the image. The object specimen image is preliminarily aligned to a reference size of 64 × 128 pixels.

  The sample point information 211 is information on sample points (sample feature points) set in each object sample image. The sample point information 211 includes a position of each sample point, a feature amount, and a sample number for specifying an object sample image in which the sample point is set.

  The feature point information 212 is feature point information created based on the sample point information 211. The content is the same as the feature point information 120 described above, and is a parameter group such as a feature point number, a relative position, a feature amount, and a variation degree of each feature point.

  The learning output unit 23 includes a USB terminal for outputting the generated feature point information 212 to the outside of the learning device 2, an interface circuit such as a CD drive and a network adapter, and respective driver programs. Each data output externally is input to the object detection apparatus 1.

  The learning control unit 22 is configured using an arithmetic device such as a DSP or MCU. The learning control unit 22 performs processing for generating feature point information 212 from the sample image 210 and outputting the generated feature point information 212 to the learning output unit 23. Specifically, the learning control unit 22 reads out and executes a program from the learning storage unit 21 and functions as a sample point setting unit 220, a clustering unit 221, and a feature point information generation unit 222, which will be described later.

  The sample point setting unit 220 functions as a sample feature point extracting unit that extracts a sample point having a predetermined image feature from the sample image. Specifically, a plurality of sample points are set in each object sample image, the feature amount at the sample point is extracted, and the position and feature amount of each sample point and the object sample image in which the sample point is set are extracted. Sample point information 211 associated with the sample number is stored in the learning storage unit 21.

  In the present embodiment, an intersection of edges called a corner or a luminance maximum point called a blob is used as a sample point. Specifically, a corner is detected from each object specimen image by a known corner detection method such as Harris-Laplace method and set as a feature point, or SIFT (Scale-Invariant Feature Transform), etc. A blob detected from each object specimen image by a known blob detection method is set as a feature point. By setting sample points having a characteristic in luminance, it is possible to efficiently generate feature point information effective for detecting an object.

  In addition, as a method of setting sample points, a method of randomly setting a plurality of sample points in a predetermined number in the entire object sample image, or a grid of sample points at equal intervals in the object sample image It is also possible to adopt a method of setting the shape.

  The sample point setting unit 220 sets an analysis window for each sample point of each object sample image and extracts a feature amount in the analysis window. The feature amount is the shape context described above in the present embodiment, but may be a HOG.

  The clustering unit 221 generates a cluster composed of sample points having similar positions and image features between the sample images. Specifically, the clustering unit 221 generates a cluster of sample points having similar positions and feature amounts by referring to the sample point information 211 and clustering the sample points by paying attention to the positions and feature amounts. As a result, the sample points representing the same part of the object among a large number of object sample images are collected into one cluster. The generated cluster information is output to the feature point information generation unit 222.

  For the clustering, known methods such as k-average clustering and various aggregation clustering methods (group average method etc.) can be used.

  At this time, the clustering may be performed while paying attention to the position and the feature amount at the same time, or the clustering focusing on the feature amount may be performed first, and then the clustering focusing on the position may be performed.

  The feature point information generation unit 222 obtains information on the distribution of sample point positions (sample distribution) by statistical analysis for each cluster, and further calculates a predetermined reference point in the sample image from a predetermined representative position in the sample distribution. The relative sample position is obtained. Then, the feature point information including the sample distribution information for each cluster and the feature point distribution information based on the sample relative position and the relative position is generated. Specifically, the feature point information generation unit 222 statistically analyzes the degree of variation in the position of the sample point for each cluster, learns the detection criterion using the feature amount of the sample point, and determines the variation degree and the detection criterion. And feature point information 212 in which the relative position of the cluster with respect to the object reference point is associated, and the generated feature point information 212 is stored in the learning storage unit 21. One feature point information 212 is generated from one cluster. In the present embodiment, the variance value of the positions of the sample points belonging to the cluster #m is calculated as the variation degree V of the feature point #m. Further, an average value of the positions of the sample points belonging to the cluster #m is calculated as a representative position of the cluster, and a vector obtained by subtracting the coordinates of the object reference point B from the calculated coordinates of the representative position is a relative position of the feature point #m. Calculate as Further, the average (average vector) of the feature amounts of the sample points belonging to the cluster #m is calculated as a detection reference for the feature point #m.

  In the configuration in which the feature point detection unit 140 of the target object detection device 1 operates as a discriminator, the feature point information generation unit 222 extracts the feature amount at the representative position of the cluster #m from each of the non-target sample images, Detection criteria by applying a learning algorithm such as Boosting or Support Vector Machine to the feature values of sample points belonging to cluster #m and the feature values extracted from the non-object sample image The discriminant function is learned.

  Note that it is unlikely that feature point information with sufficient accuracy for detecting an object can be generated from a cluster generated from a small number of object specimen images. Therefore, the feature point information generation unit 222 counts the number of object sample images that are the setting source of the sample points belonging to the cluster, and does not generate the feature point information 212 from the information of the cluster whose count value is a predetermined value or less. Can be configured. Thereby, feature point information having sufficient object detection accuracy can be generated. In addition, since it is possible to prevent an unduly small variation degree or an unduly large variation degree from being calculated, it is possible to generate feature point information having high object detection accuracy.

  FIG. 12 is a schematic diagram showing the relationship between sample points and clusters. FIG. 12 shows a state in which corners 610 to 613 ("x" marks) that are sample points having features corresponding to each other in each of the object sample images 600 to 603 are detected by being dispersed in the region indicated by the dotted line. ing. Regions 620 to 623 for extracting feature values for clustering at corners are represented by solid circles surrounding “x” marks indicating the corners. In this case, the corners 610 to 613 are collected into a cluster 650 corresponding to the dotted area. The cluster representative positions 630 to 633 (black circles) are set to the average positions of the corners collected in the cluster. The representative position becomes a feature point corresponding to the cluster, and local regions 640 to 643, which are regions for extracting feature amounts of the feature point, are represented by solid circles surrounding the feature point.

  Next, the operation of the learning device 2 will be described. FIG. 13 is a flowchart showing a schematic operation of the learning device 2. For example, when the administrator turns on the power of the learning device 2 and operates the learning operation unit 20 to instruct the start of learning, the learning device 2 performs a learning process. Hereinafter, the learning process will be described with reference to FIG.

  The learning control unit 22 sets a plurality of sample points in each object sample image by the sample point setting unit 220 (S20). Then, an analysis window is set at the position of each sample point extracted from the image in each object sample image, and a feature amount is extracted from the analysis window (S21). Sample point information 211 is generated by associating the sample number of the object sample image with the position of the sample point of the extraction source, and stored in the learning storage unit 21.

  Next, the learning control unit 22 uses the clustering unit 221 to perform clustering processing focusing on the position and the feature amount on the sample point information 211, and collects sample points having similar positions and feature amounts between the target sample images. The generated cluster is generated (S22). As a result of clustering, the clustering unit 221 adds a cluster number for identifying a cluster to which the sample point belongs to the sample point information 211 of each sample point.

  The learning control unit 22 causes the feature point information generation unit 222 to sequentially set each cluster generated in step S22 as a cluster of interest (S23), and repeats the processing of steps S24 to S29 for all clusters. Execute.

  In the cluster loop processing, the feature point information generation unit 222 refers to the sample point information 211 and calculates a representative position representing the position of the sample point belonging to the cluster of interest (S24). The relative position R with respect to the object reference point B is calculated (S25), and the variation degree V of the position of the sample point belonging to the cluster of interest is calculated (S26).

  Further, the feature point information generation unit 222 learns detection criteria using the feature amounts of the sample points belonging to the cluster of interest (S27).

  When the feature point detection unit 140 of the object detection device 1 detects a feature point by pattern matching, the feature point information generation unit 222 learns using the average feature amount of the feature points of the sample points belonging to the cluster of interest as a detection reference. In this case, the detection criterion is learned using only the image information of the object specimen image.

  In another embodiment in which the feature point detection unit 140 of the object detection device 1 operates as a discriminator, the feature point information generation unit 222 uses the feature amount of the sample point belonging to the cluster of interest, that is, the image information of the object sample image. In addition, learning is performed using the image information of the non-object specimen image. That is, the feature point information generation unit 222 also extracts the feature amount at the representative position calculated in step S24 from each of the non-object sample images, and uses the feature amount of the sample point belonging to the cluster of interest and the non-object sample image. A detection criterion is learned by applying a boosting or support vector machine to the extracted feature quantity.

  When the feature point information generation unit 222 obtains information on the feature points corresponding to the target cluster in steps S24 to S27, the new feature point number, the relative position R of the target cluster, the degree of variation V of the target cluster, and the target point New feature point information 212 is generated in association with the cluster detection criteria and stored in the learning storage unit 21 (S28).

  When all the clusters have been processed (“YES” in S29), the learning process ends.

  After completion of the learning process, when the administrator operates the learning operation unit 20 to instruct the output of the feature point information 212, the learning control unit 22 reads the feature point information 212 from the learning storage unit 21 and outputs it to the learning output unit 23. Let

  In the above embodiment, the reference point B is set at the center of gravity of the sample image 210. However, the reference point B may be at an arbitrary position such as the upper left end or the lower right end of the sample image 210 as long as it is common among the feature points. .

  In the above embodiment, the feature point information generation unit 222 calculates the variance value as the variation degree of the feature points. As another embodiment, the feature point information generation unit 222 calculates the average value of the absolute value of the difference from the average position of the sample points for each cluster as the degree of variation of the cluster, or the average position of the sample points for each cluster. The maximum value of the distance from the distance may be calculated as the variation degree of the cluster.

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Learning apparatus, 10 Imaging part, 11 Image acquisition part, 12 Detection storage part, 13 Feature point information setting part, 14 Detection control part, 15 Detection output part, 20 Learning operation part, 21 Learning storage part , 22 Learning control unit, 23 Learning output unit, 120 Feature point information, 121 Vote image, 140 Feature point detection unit, 141 Voting unit, 142 Object determination unit, 143 Abnormality determination unit, 210 Sample image, 211 Sample point information, 212 feature point information, 220 sample point setting unit, 221 clustering unit, 222 feature point information generation unit, 400,500 input image, 401 to 403 feature point, 510 to 516 voting image, 600 to 603 target object sample image, 610 613 corner, 650 clusters, 630-633 representative positions.

Claims (4)

An object detection device for detecting an object appearing in an input image,
For each of a plurality of feature points having an image feature indicating a feature of a target object image obtained by photographing the target object set in advance, a relative position between a predetermined reference point and the feature point in the target object image, and the relative position A storage unit storing feature point information including a degree of variation;
A feature point detection unit for detecting the feature points from the input image;
A voting unit that calculates a voting value with a distance characteristic according to the degree of variation around a relative position with respect to the position of the feature point in the input image for the feature point detected by the feature point detection unit;
An object determination unit that determines the presence of an object by counting the vote values at each position in the input image;
The object detection apparatus characterized by having. The object detection apparatus according to claim 1,
The object detection device according to claim 1, wherein the distance characteristic of the voting unit increases the distance range of the voting position from the center as the degree of variation increases. In the object detection device according to claim 1 or 2,
The object detection device according to claim 1, wherein the distance characteristic of the voting unit gradually attenuates the voting value in accordance with a distance away from the center as the degree of variation increases. A learning device that generates the feature point information used in the object detection device according to any one of claims 1 to 3,
A sample image storage unit storing a plurality of sample images taken of the object;
A sample feature point extraction unit for extracting a sample feature point having a predetermined image feature from each sample image;
A clustering unit for generating a cluster of the sample feature points whose positions and image features are similar between the sample images;
For each cluster, information on the sample distribution regarding the distribution of the positions of the sample feature points is obtained by statistical analysis, and further, a predetermined reference point in the sample image is a relative position from a predetermined representative position in the sample distribution. A feature point information generation unit for obtaining a sample relative position, and generating the feature point information with the information on the sample distribution and the sample relative position for each cluster as the feature point distribution information and the relative position;
A learning apparatus comprising:

Images