JP2011210181A - Learning device and object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the following problem of an object detection device based on an image: when detecting a portion of an object so as to cope with concealment, high-accuracy detection in a partially concealed state is difficult from the relation of a trade-off between identification performance and concealment resistance corresponding to size of the set portion.SOLUTION: A detection storage part 12 stores information about the portion preset by learning as portion information 120. A portion detection part 141 identifies presence/absence of the portion corresponding to the portion information in each position of an input image, and outputs a position identified that there is the portion. An object decision part 142 detects the object when the outputted positions concentrate by a threshold value or above. In the portion, an identification rate is set to a minimum magnitude exceeding a target value.

Description

  The present invention relates to an object detection device that detects an object imaged in an input image, and a learning device used for learning thereof.

  In recent years, active research has been conducted to detect the presence of people, faces, and the like from images from surveillance cameras and digital still cameras. In the detection process, a search method using a pattern matching device or a classifier is used. That is, windows are set at various locations in the image, the window images are input to a pattern matching device and a discriminator, and the detection results output by these are totaled to detect an object at a position where the total value is high.

  The object of the image is not necessarily captured as a whole, and a part of the object may be concealed by another object. In order to detect an object in a partially concealed state, conventionally, the object is divided into a plurality of parts, each part is detected, and the detection results of these parts are integrally determined.

  For example, in the prior art described in Patent Document 1, a relatively small block having a predetermined size is set as a target part. Further, there is a conventional technique for setting a relatively large part such as a head, a torso, and a leg for a person who is an object.

JP-A-9-21610

  The site setting in the prior art has been performed without considering the tradeoff between detection reliability and concealment resistance caused by the size of the site.

  That is, when the part is enlarged, the reliability of the detection result is high when the part can be detected, but the part is easy to be concealed, so the concealment resistance is low and the possibility of detection omission increases. There's a problem.

  Conversely, if the part is made smaller, concealment is difficult and detection omission of the part is reduced, but detection by chance is established with an image other than the part because information used for matching is reduced. There is a problem that the possibility of erroneous detection increases and the reliability of the detection result decreases.

  Further, the relationship between reliability and size differs depending on the part of the detection target. Therefore, in the conventional part setting, there is a problem that the detection result of each part does not have uniform reliability, and the reliability of the integrated determination changes depending on the concealment situation.

  As described above, in the conventional technology, the size of the part is set without considering the trade-off between detection reliability and concealment tolerance, so a target that is partially concealed can be detected with high accuracy. It was difficult to do.

  The present invention has been made to solve the above-described problems, and provides an object detection device capable of detecting an object in a partially concealed state with high accuracy, and a learning device used for constructing the object detection device. The purpose is to do.

  A learning apparatus according to the present invention generates information on a part of an object used for detection of the object, and stores a sample image of the object and a sample image of a non-object in advance. An image storage unit, a region candidate generation unit that sets a predetermined region reference point in the specimen image, sequentially generates regions that include the region reference point and have different sizes, and the sample for each region A learning unit that learns an identification criterion for identifying the presence / absence of the part using at least an image feature of the part in the specimen image of the object in the image, and each specimen image according to the identification reference of each part An identification rate determination unit that determines the identification rate by identifying the presence / absence of the region, determines that the region where the identification rate exceeds a predetermined target value is valid, and among the regions determined to be effective by the identification rate determination unit It comprises the minimum of site serial magnitude, the site information generating unit that generates a site information including said identification criteria of the site, the.

  In another learning device according to the present invention, the part candidate generation unit sets a sample frame of the size surrounding the part reference point, and generates a partial region or a whole region in the sample frame as the part. To do.

  In another learning device according to the present invention, the part candidate generation unit sets a plurality of local regions, which are analysis units of the image features, in the sample image, and increases or decreases the number of the local regions surrounded by the sample frame. Thus, the size is controlled, and a collection of the local regions surrounded by the sample frame is generated as the part.

  In a preferred aspect of the learning apparatus according to the present invention, the region candidate generation unit subdivides the sample image at predetermined equal intervals to set the local region.

  In another preferred aspect of the learning apparatus according to the present invention, the part candidate generation unit extracts a plurality of feature points of the target object from the sample image of the target object, and the feature points are extracted at positions where the feature points are extracted. The local area is set.

  In still another preferred aspect of the learning device according to the present invention, the part candidate generation unit generates a random point in the sample image, and sets the local region at a position where the random point is generated. .

  In another preferable aspect of the learning device according to the present invention, the part candidate generation unit sets the part reference point at each position where the local region is set.

  The target object detection apparatus according to the present invention detects the target object captured in an input image using the information generated by the learning apparatus, and is generated by the part information generation unit. A part information storage unit for storing part information and the presence / absence of the part corresponding to the part information stored in the part information storage unit at each position of the input image are identified, and the part is identified. A part detection unit that outputs a position in the input image, and an object determination that detects the object when the position output by the part detection unit is concentrated beyond a preset object detection criterion A section.

  According to the learning device according to the present invention, it is possible to construct the target object detection device capable of detecting a target object in a partially concealed state with high accuracy, and the target object detection device according to the present invention is partially An object in the concealed state can be detected with high accuracy.

It is a block diagram which shows the structure of the outline of the target object detection apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows an example of the relationship of the local region, the site | part, and target object which were produced by the feature point sampling. It is a schematic diagram which shows an example of the relationship of the local area | region, site | part, and target object which were produced by grid sampling. It is explanatory drawing which shows the specific example of site | part information typically. It is a schematic diagram explaining the example of the site | part detection process by a site | part detection part. It is a flowchart which shows the operation | movement of the outline of the target object detection apparatus which concerns on embodiment of this invention. It is a general | schematic flowchart of the site | part detection process in the target object detection apparatus which concerns on embodiment of this invention. It is a block diagram which shows the schematic structure of the learning apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the relationship between the feature point in a feature point sampling method, or the random point in a random sampling method, a cluster, and a sample point. It is a schematic diagram which shows the step of the magnitude | size of the sample area | region in each of the feature point sampling method, the random sampling method, and the grid sampling method. It is a flowchart which shows the operation | movement of the outline of the learning apparatus which concerns on embodiment of this invention.

  Hereinafter, an object detection device 1 and a learning device 2 which are embodiments of the present invention (hereinafter referred to as embodiments) will be described with reference to the drawings. The target object detection apparatus 1 detects a target object captured in the input image by detecting a part of the target object from the input image. In this embodiment, a person is an object, and a monitoring image obtained from the monitoring space is an input image. The object detection device 1 detects an intruder by detecting a human part from the monitoring image, and outputs an abnormal signal when the intruder is detected. The learning device 2 generates part information used for the object detection device 1 by learning, and specifically generates a region setting and a classifier corresponding to the part used for the object detection device 1.

[Object detection device]
FIG. 1 is a block diagram illustrating a schematic configuration of an object detection device 1 according to the embodiment. The object detection device 1 includes an imaging unit 10, an image acquisition unit 11, a detection storage unit 12, a part information setting unit 13, a detection control unit 14, and a detection output unit 15. The image acquisition unit 11 is connected to the imaging unit 10, and the image acquisition unit 11, the detection storage unit 12, the part information setting unit 13, and the detection output unit 15 are connected to the detection control unit 14.

  The imaging unit 10 is a surveillance camera and is installed in a surveillance space. For example, the monitoring camera is installed on the ceiling of the monitoring space over the monitoring space. The monitoring camera images the monitoring space at a predetermined time interval (for example, 1 second), and sequentially outputs monitoring images in which each pixel is expressed by a multi-gradation pixel value.

  The image acquisition unit 11 is an interface circuit that acquires a monitoring image captured by the imaging unit 10 and imports the monitoring image into the detection control unit 14. Hereinafter, an image input from the image acquisition unit 11 to the detection control unit 14 is referred to as an input image.

  The detection storage unit 12 is a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk, and stores programs and data used by the detection control unit 14. The detection storage unit 12 inputs and outputs these programs and data to and from the detection control unit 14. The data stored in the detection storage unit 12 includes part information 120 and part detection information 121.

  A site | part is a part of target object, and the some site | part is preset with respect to the target object. The part information 120 is information about each of a plurality of parts of the object, and is a part number for specifying the part, a region setting that determines the size and shape of the part, and an identification for identifying the presence or absence of an image feature of the part The relative position (part relative position) of the part from the part reference point with respect to the reference and the reference point (object reference point) set in common to the plurality of parts. The positional relationship between the parts via the object reference point is defined by the part relative position.

  This part information 120 is created in advance by the learning device 2 described later using a specimen of an object image (object specimen image) and a specimen of a non-object image (non-object specimen image). In particular, the identification criterion for each part is a part identification criterion for identifying the presence or absence of a partial image of the target part of the target object in the comparison area in the input image. The image information of the target part in the target specimen image and the non-target specimen It is determined that the identification rate for identifying the image information of the contrast area in the image exceeds a preset target value ε.

  Here, there is a relationship between the size of the part to be identified by the identification standard and the identification rate of the identification standard, basically that the larger the part, the higher the identification rate. Therefore, when a target value of the identification rate is set, a part having a size corresponding to the target value can be determined. On the other hand, the smaller the size of the part, the more difficult it is to conceal it. Each part of the part information 120 is given a relationship of a minimum size at which the identification rate exceeds the target value between the identification standard and the size determined by the region setting. In the present embodiment, each part is set to have a minimum size that exceeds the target value of the identification rate when sequentially enlarged by a certain procedure. Thereby, the reliability of the detection result concerning each part and the resistance to concealment of detection are compatible, and highly accurate object detection becomes possible.

  An image feature analysis unit used for identification is a local area smaller than each part, and each part is a collection of a plurality of local areas. The image feature used for identification is a feature quantity such as a well-known shape context (Shape Context) or histogram of oriented gradients (HOG). The positional relationship between the local regions constituting the region is determined in the region setting of the region information 120, and the positional relationship close to each other is set. A compact region is configured by collecting local regions having such a positional relationship.

  The shape context is a feature amount representing the distribution characteristic of the edge in the local region, and the data is in a vector format. The index of each element of the vector corresponds to a combination of a small area divided into a plurality of local areas and the quantized edge direction, and the value of each element has the edge direction represented by the index in the small area represented by the index Corresponds to the sum of edge strengths. HOG is a feature amount representing the distribution characteristic of the luminance differential value in the local region. Both the shape context and the HOG represent the distribution characteristics of the luminance gradient in the local region, and are suitable for detecting an object because they are robust against illumination fluctuations.

  FIG. 2 is a schematic diagram showing an example of a relationship between a local region, a part, and an object created by feature point sampling described later. In this example, a shape context is used as an image feature, and the local region is circular. FIG. 3 is a schematic diagram showing an example of the relationship between local regions, parts, and objects created by grid sampling described later. In this example, HOG is used as an image feature, and the local area is rectangular.

  2 (a) and 3 (a) show examples of a local region, a part, etc. on a part of an object specimen image represented by an alternate long and short dash line by a person who is an object. Yes. In FIG. 2A, each solid circle is a local region, a collection of the local regions is a part, and a dotted line is a schematic shape of the part. When feature point sampling is used, the outline of a part generally has a shape that can be called a cloud shape in which unevenness of a circle in a local region appears, and the approximate shape does not necessarily have a regular shape such as a circle or an ellipse. In FIG. 3A, the bold rectangle is a part, the black circle is a lattice point (grid), and each block obtained by dividing the part by a thin line is a local region grouped as the part. One local region is formed around one lattice point, and a plurality of local regions are combined to form a region.

  FIGS. 2B and 2C and FIGS. 3B and 3C are diagrams showing a part and the like on the object specimen image, and the alternate long and short dash line schematically represents the object. Here, each object specimen image is standardized into a vertically long rectangle of 64 pixels in the width (horizontal) direction × 128 pixels in the height (vertical) direction according to the shape of the person, and the barycentric coordinates (32, 64) thereof. It is defined as the object reference point B.

  Each of a number of solid-line ellipses in FIG. 2B and each of a plurality of thick-line rectangles in FIG. 3B corresponds to a portion created for the object. In FIG. 2 (b) and FIG. 3 (b), when comparing the sizes of a plurality of parts set on the object, there is a small part around the head that is relatively complex and characteristic in shape. A large portion is set around the leg portion having a relatively simple shape and few features.

Here, the number of set parts is M (> 1), and a part number of 1 to M is given through each part. The positions of the M parts are different from each other, and for each of the parts # 1 to #M, vectors R _{1 to} R _M from one point (part reference point) in the part to the object reference point B are relative to the part of the part. It is stored in the part information 120 as the position. In this example, one of the N _m local regions constituting the part #m is defined as a reference local area, and the center of the reference local area is set as a part reference point. Incidentally, the index of the 1 to N _m is assigned to each site in each local region, and the index of the reference local regions 1.

Area setting information is associated with each of the M parts, and the information is stored in the part information 120. The area setting information of the part #m includes the number N _{m of} local areas constituting the part and the relative positions (local area relative positions) L _{m, 1 of} the N _m local areas with reference to the reference local area. ~ Lm _{, Nm} .

The solid oval in FIG. 2 (c) represents the part (part #m) shown in FIG. 2 (a), and the two small circles in the ellipse are reference local areas among the local areas constituting the part #m. And the nth local region. Hereinafter, the n-th (1 ≦ n ≦ N _m ) local region of the part #m is represented as # m, n.

  The thick line rectangle in FIG. 3C represents the part #m shown in FIG. 3A, and the two small thin line rectangles in the rectangle are the reference local area and n among the local areas constituting the part #m. Th local region.

In either of FIGS. 2 (c) and 2 FIG. 3 (c), the site relative position R _m site #m is defined by the vector from the center of the reference local regions to an object reference point B, also local area #m , n local region relative position L _{m, n} is a vector from the center of the reference local region to the center of the local region # m, n.

In this embodiment, the identification criteria stored in the part information 120 are classifiers H ₁ (x) to H _M (x) for the parts # 1 to #M. x is the feature quantity of the input image input to the classifier. The classifier H _m (x) of the part #m is a so-called strong classifier, and has been learned with respect to each feature amount of the N _m local regions constituting the part #m as shown in the following formula (1). It is learned as a linear sum of weak classifiers h ₁ (x ₁ ) to h _Nm (x _Nm ). The classifier for each part is learned using the feature amount of the part in the object specimen image and the feature amount of the contrast area in the non-object specimen image. Learning is performed by a boosting method such as Adaboost or a machine learning method such as a support vector machine.

  All of the M classifiers are determined by the identification rate determination unit 222 of the learning device 2 to be described later that the identification rate exceeds a preset target value ε.

  In another embodiment, a template for each part is set as the identification criterion.

  In the above description, various information stored in the part information 120 has been mentioned. FIG. 4 is an explanatory diagram schematically showing a specific example of the part information 120.

  The part detection information 121 stores information of a result (identification result) obtained by performing a part detection process on the input image by a part detection unit 141 described later. In the present embodiment, since detection processing is performed at a plurality of magnifications (detection magnifications), the content of the part detection information 121 includes the detection magnification. That is, as the part detection information 121, data is stored that includes the detection position of each part in the input image, the part number of the part, the degree of detection of the part, and the detection magnification of the part.

  The part information setting unit 13 is an interface circuit such as a USB terminal, a CD drive, and a network adapter that inputs the part information 120 from the outside, and each driver / program, and a program that stores the input part information 120 in the detection storage unit 12 Consists of. The part information 212 generated by the learning device 2 is input via the part information setting unit 13 and stored as part information 120 in the detection storage unit 12.

  The detection control unit 14 is configured using an arithmetic device such as a DSP (Digital Signal Processor) or an MCU (Micro Control Unit). The detection control unit 14 processes the input image from the image acquisition unit 11 to determine the presence or absence of a person, and performs processing to output an abnormal signal to the detection output unit 15 when a person is detected. Specifically, the detection control unit 14 reads out and executes a program from the detection storage unit 12, and functions as a detection magnification change unit 140, a part detection unit 141, an object determination unit 142, and an abnormality determination unit 143 described later.

The detection magnification change unit 140 adjusts the detection magnification to adapt to the variety of object sizes when detecting each part in response to the various sizes of the objects captured in the input image. Process. Here, the detection magnification α is the size of the object captured in the input image when the size of the object captured in the object specimen image (that is, the size of the object represented by the part information 120) is used as a reference. It is a magnification of the size. Specifically, the detection magnification changing unit 140 enlarges or reduces the input image in accordance with a plurality of preset detection magnifications. By the enlargement / reduction, the input image becomes 1 / α of the original size. The detection magnification α is, for example, (1.05) ³ times, (1.05) ² times, 1.05 times, 1.0 times, 1 / 1.05 times, 1 / (1.05) ² times, 1 /(1.05) Set to 7 levels, ³ times. Enlarging / reducing processing can be performed by a known bilinear interpolation method or the like.

  The part detection unit 141 sets a detection frame corresponding to the region setting of the part for each part in each position of the input image, compares the image feature in the detection frame with the identification reference, detects the part, It is a site | part determination part which produces | generates the site | part detection information 121 containing a site | part number and a detection position. The part detection information 121 includes the detection degree and detection magnification of the part as necessary. The generated part detection information 121 is input to the object determination unit 142.

Specifically, the part detection unit 141 performs a scanning process in which a detection frame that is a congruent figure with a part is sequentially set at each position in the input image. As described above, the site is a collection of a plurality of local regions. The part detection unit 141 sets detection frames corresponding to a plurality of local regions constituting the part at each position, extracts the feature amount in each frame, and uses the extracted feature quantity as a classifier H _{m for the part.} Input to (x) and calculate the likelihood (detection degree) which is the degree of likelihood that the image in the detection frame is the part of the object. The part detection unit 141 compares the likelihood with a preset part detection threshold value Tp, and when the likelihood exceeds Tp, the part detection unit 141 detects the part at that position.

Note that the feature quantity extracted from the local region # m, i of the part #m is x _i on the right side of the expression (1), and the output of the classifier H _m (x) represented by the expression (1) is the part #. The likelihood of m. H _m (x) outputs a positive value at the position where the part #m is imaged in the input image. Correspondingly, the part detection threshold Tp is set to 0. Note that, as described above, the boundary value of the likelihood of the target / non-target is 0 for a general classifier, but a classifier that sets a boundary value other than 0 has also been proposed. When such a discriminator is used, Tp is set to the boundary value.

  In another embodiment in which a template is used as an identification criterion, the degree of coincidence (detection degree) is calculated by matching processing between the template and the feature quantity in the detection frame, and the degree of coincidence is set in advance for the degree of part detection. A site is detected when the threshold is exceeded.

  FIG. 5 is a schematic diagram for explaining an example of the part detection processing by the part detection unit 141. FIG. 5A shows a state in which the part detection unit 141 detects the part #m of FIG. 2A created by the feature point sampling by raster scanning the input image. Each of a plurality of white circles enlarged and shown in the dotted ellipse is a frame corresponding to the local region of the part #m, and a set of these is a frame corresponding to the part #m. As described above, the set of white circles generally forms a cloud-shaped outline, and the cloud-shaped figure becomes the outline of the frame. Black circles are positions (scanning positions) set in the input image. The center of each local area is calculated by shifting from the scanning position by the local area relative position L, and a frame having the same shape and size as the local area is set at the calculated center position of the local area. In the example shown in FIG. 5A, a likelihood greater than the part detection threshold Tp is calculated at a position (x1, y1) or (x2, y2) where the object is imaged, and the part #m is detected.

  In FIG. 5B, the part detection unit 141 raster scans the same input image as in the example of FIG. 5A, and the part #m of FIG. 3A created by grid sampling is extracted from the input image. The state of detection is shown.

  The object determination unit 142 integrates the part detection information 121 of each part generated by the part detection unit 141 to determine whether or not the target object image exists in the input image, and sends the determination result to the abnormality determination unit 143. Output. The object determination unit 142 determines that the object is captured in the input image when the detection positions are concentrated beyond a preset object detection criterion. Specifically, the object determination unit 142 calculates a relative object reference point (relative reference point) that is shifted from the detection position of each part by the relative position of the part. Then, the number of the part detection information 121 whose relative reference points substantially coincide with each other is totaled and compared with a preset object detection threshold value To, and it is determined that there is an object at the relative reference point whose total value exceeds To. On the other hand, if there is no relative reference point whose total value exceeds To, it is determined that there is no object in the input image.

If the detection position of the part #m is a position vector P and the detection magnification at the time of detection is α, the relative reference point position vector Q is calculated as Q = (P + R _m ) / α. R _m is a vector representing the relative position of the part #m as described above.

  The process of converting the detected position of a part into a relative reference point is called voting because it is similar to casting one vote for each detection result. Instead of summing up the number of part detection information 121, weighted summation may be performed in which the sum of the degree of detection of part detection information 121 whose relative reference points substantially match is calculated.

  In addition, aggregation may be performed for each detection magnification, but because the voting value of the same object may be set across multiple detection magnifications due to individual differences in the imaging state and proportion of the object, the detection magnification It is preferable to allow the tabulation in the adjacent part detection information 121.

  When the object determination unit 142 determines the presence of the object, the abnormality determination unit 143 outputs an intrusion abnormality signal to the detection output unit 15 assuming that an intrusion abnormality is detected.

  The detection output unit 15 is an interface circuit that is connected to an external device and outputs an intrusion abnormality signal to the external device. The external device is an alarm display means such as a speaker, a buzzer, or a lamp for alarming the presence of an intruder, a remote center device connected via a communication network, and the like.

  Next, operation | movement of the target object detection apparatus 1 is demonstrated. FIG. 6 is a flowchart showing a schematic operation of the object detection apparatus 1. For example, when the device administrator turns on the power, each unit starts operating. The image acquisition unit 11 inputs images captured at predetermined time intervals to the detection control unit 14. The detection control unit 14 repeats the process consisting of steps S10 to S19 each time an image is input.

  When an image is input (S10), the part detection unit 141 of the detection control unit 14 detects each part from the input image (S11).

  FIG. 7 is a schematic flowchart of the part detection process S11. The site detection process S11 will be described with reference to FIG.

  The part detection unit 141 sequentially sets the seven detection magnifications to the attention magnification (S110), and executes a loop process that repeats the processes of steps S111 to S121 for all the detection magnifications.

  In the loop processing of the detection magnification, first, when the attention magnification is other than 1, the part detection unit 141 generates an input image having a size corresponding to the attention magnification by performing enlargement or reduction (S111). The part detection unit 141 sequentially sets all pixel positions of the input image to the center of a frame having the same shape and size as the local region, and extracts a feature amount in the frame at each set position (S112). . The extracted feature amount is associated with the extraction position and temporarily stored in the detection storage unit 12 as feature amount information. By calculating and storing the feature amount at this stage and making it available at any time in later processing, it is possible to eliminate unnecessary duplication calculation.

  The part detection unit 141 sequentially sets M parts #m (1 ≦ m ≦ M) stored in the part information 120 as attention parts (S113), and further sequentially sets each pixel position in the input image, A loop process that repeats the processes of steps S115 to S120 is executed for all combinations of the part and the pixel position, which is set as the position of interest (S114).

  In the loop processing relating to the part and the pixel position, first, the part detection unit 141 refers to the part information 120 of the target part, sets a detection frame of the target part at the target position (S115), and further sets an identification criterion for the target part. The degree of detection is calculated in comparison with the feature amount in the detection frame (S116).

Feature amount x _i at each position of the input image have already extracted in step S112. Therefore, in step S116, the part detection unit 141 calculates the center of each local region (that is, the center of each detection frame) constituting the target part by adding each local region relative position of the target part to the target position. The feature quantity x _i having the obtained center as the extraction position is read from the feature quantity information generated and stored in step S112, and input to the classifier H _m (x) of the target site. The likelihood output from the discriminator is the degree of detection.

  The part detection unit 141 compares the obtained degree of detection with the part detection threshold Tp (S117). If the degree of detection exceeds Tp (“YES” in S117), the part of interest is detected at the target position. As shown, the part detection information in which the attention magnification, the part number of the part of interest, the position of attention, and the detection degree are associated is generated and stored in the detection storage unit 12 (S118). On the other hand, when the degree of detection is equal to or less than Tp (“NO” in S117), step S118 is omitted because no target region is detected at the target position.

  Thus, when the entire input image has been scanned for all the parts and all the detection magnifications (“YES” in S119, “YES” in S120, and “YES” in S121), the part detection process S11 ends.

  When the part detection process S11 ends, as shown in FIG. 6, the process of the object detection apparatus 1 proceeds to the vote value totaling process S12. In the vote value counting process S12, the object determination unit 142 performs voting based on the part detection information and counting of the voting results as described below (S12).

  The object determination unit 142 prepares a voting image having the same size as the original input image for each detection magnification, and clears these pixel values to zero. Next, the object determination unit 142 reads the part relative position R of the part corresponding to the part number of each part detection information from the part information 120, and adds the part relative position R to the detection position of the part detection information. The relative reference position is calculated by multiplying the result by the detection magnification of the part detection information. Subsequently, the object determining unit 142 adds the degree of detection of the part detection information to the pixel value of the relative reference position in the voting image corresponding to the detection magnification of each part detection information. Furthermore, the object determination unit 142 adds pixel values at positions corresponding to each other between the vote images having adjacent detection magnifications for each detection magnification.

  When voting and counting (S12) are completed in this way, the object determination unit 142 divides the voting image at each detection magnification into blocks, and detects the peak pixel having the maximum pixel value (total value) for each block (S13). ). As the block size, a size smaller than the size obtained by multiplying the size of the object specimen image by (1-maximum allowable concealment rate) is set in advance.

  The object determination unit 142 sequentially sets each peak pixel as a target peak pixel (S14), and executes a loop process that repeats the processes of steps S15 to S17 for all peak pixels. In this peak pixel loop processing, the object determination unit 142 compares the pixel value (total value) of the peak pixel of interest with the target detection threshold To (S15), and if the total value is greater than To, If the object is detected at the position (“YES” in S15), the object detection information in which the position of the peak pixel of interest, the pixel value of the peak peak pixel, and the detection magnification of the voting image to which the peak peak pixel belongs is associated. Generated and stored in the detection storage unit 12 (S16). On the other hand, when the total value is equal to or less than To (“NO” in S15), step S16 is omitted.

  When the processing has been completed for all the peak pixels in this way (“YES” in S17), the processing of the object determination unit 142 ends.

  When the object determination unit 142 finishes the process, the abnormality determination unit 143 of the detection control unit 14 refers to the detection storage unit 12 to confirm the presence / absence of the object detection information (S18), and even one object detection information exists. If it is stored, the object is detected ("YES" in S18), an intrusion abnormality signal is output to the detection output unit 15, and an alarm is output to the detection output unit 15 (S19).

  When the above process is completed, the process returns to step S10 again.

  In the above embodiment, the image acquisition unit 11 is connected to the imaging unit 10, and the detection control unit 14 detects the object by online processing. However, the image acquisition unit 11 may be connected to the recording device, and the detection control unit 14 may detect an object by offline processing.

  In the above-described embodiment, the detection magnification changing unit 140 enlarges / reduces the input image according to the detection magnification, and the part detection unit 141 sets a detection frame congruent with the part. However, the detection magnification changing unit 140 does not enlarge or reduce the input image, but instead the part detection unit 141 sets a detection frame for a similar figure of a part having a detection magnification as a similarity ratio at each position in the input image. It can also be.

  In the above-described embodiment, the part detection unit 141 sets detection frames at all the pixel positions in the input image. However, the part detection unit 141 is set only in the possible range of the target object set in advance in the input image according to the environment of the monitoring space. A detection frame may be set. In still another embodiment, the part detection unit 141 may first extract feature points from the input image, and set a detection frame only within the peripheral range of the extracted feature points. In addition, the part detection unit 141 stores the feature amount of the reference local region of each part in the part information 120 of the part, and first matches the feature amount with all the pixel positions in the input image. A detection frame may be set only at the pixel position at which is obtained.

[Learning device]
FIG. 8 is a block diagram illustrating a schematic configuration of the learning device 2 according to the embodiment. The learning device 2 includes a learning operation unit 20, a learning storage unit 21, a learning control unit 22, and a learning output unit 23. The learning operation unit 20, the learning storage unit 21, and the learning output unit 23 are connected to the learning control unit 22.

  The learning operation unit 20 is a user interface device such as a keyboard and a mouse, and is operated by an administrator of the device, and gives a learning start instruction and a part information output instruction to the learning control unit 22.

  The learning storage unit 21 is a storage device such as a ROM, a RAM, and a hard disk, and stores programs and data used by the learning control unit 22. The learning storage unit 21 inputs and outputs these programs and data to and from the learning control unit 22. The data stored in the learning storage unit 21 includes a specimen image 210, part candidate information 211, and part information 212.

  The sample image 210 is an image serving as a basis for creating the part information 212, and is stored in advance prior to the learning. The sample image 210 includes a large number (several thousands to several tens of thousands) of target object images in which the target is imaged, and a large number (several thousands to tens of thousands) of the non-target samples in which the target is not captured. It consists of an image. Each of the sample images 210 is given a sample number for identifying the image. Each sample image 210 is pre-aligned to a reference size of 64 × 128 pixels.

  The part candidate information 211 is information about each part created and learned from the sample image 210. The part candidate information 211 is information that becomes a candidate for part information 212 to be described later, and is distinguished from the part information 212 itself that becomes part information 120 of the object detection device 1 described above. However, the content conforms to the part information 120 and 212. Specifically, the part candidate information 211 includes the part number of each part, the region setting of the part, the identification criterion of the part, and the part relative position of the part. Etc.

  The part information 212 is information selected from the part candidate information 211. The contents are the same as the part information 120 described above, and are the part number of each part, the region setting of the part, the identification criterion of the part, and the part relative position of the part.

  The learning output unit 23 includes a USB terminal for outputting the generated part information 212 to the outside of the learning device 2, a CD drive, an interface circuit such as a network adapter, and respective drivers and programs. Each data output externally is input to the object detection apparatus 1.

  The learning control unit 22 is configured using an arithmetic device such as a DSP or MCU. The learning control unit 22 performs a process of generating the part information 212 from the sample image 210 and outputting the generated part information 212 to the learning output unit 23. Specifically, the learning control unit 22 reads and executes a program from the learning storage unit 21 and functions as a part candidate generation unit 220, a learning unit 221, an identification rate determination unit 222, and a part information generation unit 223, which will be described later.

  The part candidate generation unit 220 sets a plurality of different part reference points in the object specimen image, and generates and generates a part including the part reference point for each part reference point by sequentially changing the size. The learned part is input to the learning unit 221. What is generated here is a candidate for a part, and the generated candidate is used for selection in the subsequent processing. The part candidate generation unit 220 generates part candidates having different sizes, so that part information having both concealment resistance and identification performance can be generated. Note that the size of the part is set to be larger than one local region and smaller than the reference size of the specimen image 210. For example, the upper limit of the size is set to 1/2 of the reference size.

  As described above, one or more candidates are generated for one part reference point by generating part candidates that include the part reference point and have different sizes. As a result, it becomes easy to generate part information of a part having a part that is not very characteristic in the target specimen image as a part reference point, and the coverage with respect to the target specimen image is improved. Thereby, it becomes possible to adapt to various concealment states of the object.

  Also, by setting a plurality of different site reference points, the completeness with respect to the object specimen image is improved, and it becomes possible to adapt to various concealment states of the object.

  In order to change the size of the part, specifically, the part candidate generation unit 220 sets a local region, which is an image feature analysis unit, at a plurality of positions in the object specimen image, and sets a part reference point. The surrounding sample frames are sequentially changed in size and set, and a region in which local regions in the sample frame are collected is generated. That is, the part candidate production | generation part 220 produces | generates the one part area | region or all area | region in a sample frame as a part. In this way, by limiting the size of the part by the size of the sample frame, it is possible to reliably and easily control the size of the part while collecting adjacent local regions. In particular, in the case of the feature point sampling method or the random sampling method described above, since the local regions are discretely arranged, the effect of introducing the sample frame is high. At this time, in particular, the part candidate generation unit 220 changes the size of the sample frame stepwise by changing the number of local regions surrounded by the sample frame. Thereby, when the size of the part is changed, the identification rate of the identification reference can be changed reliably and easily, and the efficiency of the part information generation can be increased.

  As a method of setting the size step, it is possible to set the sample frame consisting of one local region as the minimum step and to determine that the number of local regions sequentially increases in a relatively small step. This is preferable from the viewpoint of efficient generation.

  The site candidate generation unit 220 sets a plurality of sample points in the object sample image according to a predetermined standard, sets a local region around these sample points, and sets each sample point as a site reference point. When the part reference point and the sample point are shared, the local region is treated as the reference local region described above.

  Hereinafter, three types of setting methods for sample points and local regions will be described.

<Feature point sampling method>
A feature point is detected from the object sample image, and a representative position of each detected feature point is set as a sample point. As the feature point, an intersection of edges called a corner or a luminance maximum point called a blob is used. Specifically, a corner is detected as a feature point from each object specimen image by a known corner detection method such as the Harris-Laplace method, or a known blob such as SIFT (Scale-Invariant Feature Transform). A blob is detected as a feature point from each object specimen image by the detection method.

  Here, feature points detected for the same part from a large number of object specimen images are not detected at exactly the same position. Therefore, the part candidate generation unit 220 extracts feature amounts from the feature points of each target specimen image, and performs clustering focusing on the positions of the feature points and the feature quantities between the target specimen images to generate feature point clusters. Then, the average position of the feature points belonging to each cluster is determined as the sample point of the local region.

  FIG. 9 is a schematic diagram showing the relationship between feature points in the feature point sampling method or random points, clusters, and sample points in the random sampling method described later. FIG. 9 shows a state in which feature points (or random points) 31 (x marks) corresponding to each other in each object specimen image 30 are detected in a region 32 indicated by a dotted line. In this case, the region 32 corresponds to a cluster of feature points (or random points), and the average position of the feature points (or random points) 31 is set to the sample point 33 (black circle). A solid line circle centered on the sample point 33 represents a local region set corresponding to the sample point 33. Note that a solid circle surrounding the feature point (or random point) 31 represents a region from which a feature amount for clustering is extracted.

  The sample point may be the median or mode of the position of the feature point belonging to each cluster, instead of the average position of the feature point belonging to each cluster.

  Since feature points can be easily obtained with feature points that are significant for part identification, the part that is generated by collecting local regions based on the feature points increases and decreases more reliably at each size step, so the part information can be efficiently obtained. Can be generated.

<Grid sampling method>
A plurality of sample points are set at equal intervals on the entire object sample image, and a local region centered on each sample point is set.

  The number of parts using the local region by this method can be controlled by the interval between the sample points. It may be set to allow overlap between local regions. This method is equivalent to dividing the object specimen image at equal intervals.

  In the feature point sampling method described above, feature points may not be detected evenly over the entire object, but with the grid sampling method, parts can be set evenly over the whole object, so that parts that can detect objects in various concealed states can be detected. Information can be generated reliably.

<Random sampling method>
A plurality of sample points are set at random on the entire object sample image, and a local region centered on each sample point is set.

  The number of parts using the local region by this method can be controlled by the number of sample points generated at random.

  Similarly to the grid sampling method, the random sampling method can uniformly set a part over the entire object, and can reliably generate part information that can detect objects in various concealed states.

  The method for setting three types of sample points and local areas has been described above. FIG. 10 is a schematic diagram showing the stage of the size of the sample frame in each method. FIG. 10A shows a state in which the feature point sampling method and the random sampling method are employed, and shows the size of the four sample frames for a certain sample point. FIG. 10B shows a state in which the grid sampling method is employed, and shows the size of the sample frame for a certain sample point in three stages. In FIG. 10, a black circle represents a sample point, a thin line circle or rectangle surrounding the sample point represents a local region, a thick line circle or rectangle represents a sample frame, and a hatched local region represents a reference local region. Sample points in the reference local region are site reference points.

  In the learning by the boosting method that is added while selecting the feature amount, the sample frame is set to include a plurality of local regions at the minimum stage that is the initial state, and a plurality of specimen frames are increased when the stage is enlarged by one stage. By setting in this way, there is always room for selecting a feature amount, and significant learning can be performed. From this point of view, in this embodiment, for example, the minimum sample frame includes at least five local regions, and is set so that the number of local regions increases by three or more when enlarged by one step.

  The learning unit 221 (identification reference generation unit) learns the identification reference using the image feature of the part in the sample image 210. That is, the learning unit 221 extracts the image feature of the part in the target specimen image and the image feature of the part in the non-target specimen image for the part input from the part candidate generation unit 220, and adds the booth to these image features. Apply the Ting method to learn the classifier. In another embodiment using a template as an identification criterion, the learning unit 221 learns the template by averaging the image features of the part in the object specimen image. In the learning in this case, the non-object sample image is not necessary. Part candidate information 211 is generated and stored by learning of the identification criteria in the learning unit 221.

The identification rate determination unit 222 obtains an identification rate based on the identification criterion of the part learned by the learning unit 221 and stored as the part candidate information 211, and determines that the part whose identification rate exceeds the target value is effective. Specifically, the identification rate determination unit 222 calculates the identification rate by inputting the object specimen image and the non-object image to the classifier of the part stored in the part candidate information 211, and the identification rate is the target. It is determined whether or not the value ε is exceeded. Discrimination rate determination unit 222 inputs the feature value x _i which is extracted from the local region within the site of the object specimen image identifier H is determined that an error if the output is below site detection threshold Tp, its output further feature amount x _i which is extracted by setting the region to the non-object image is input to the identifier H is determined that an error greater than Tp. The error rate is calculated by counting the number of these errors and dividing the count value by the total number of sample images 210. If the error rate is less than the target value, it is determined that the identification rate has exceeded the target value. The target value for the error rate is set within a range of 0 or more and less than 0.5. The target value setting is common to all parts. Note that the accuracy rate may be calculated instead of the error rate, and if the accuracy rate is larger than the target value, it may be determined that the identification rate has exceeded the target value.

  In the case of a configuration in which a template is learned as an identification reference, the template matching result is determined instead of the process of determining the output of the classifier. That is, the identification rate determination unit 222 calculates the degree of coincidence between the feature quantity extracted from each local region in the region of the object specimen image and the template and determines that the degree of error is equal to or less than Tp. The degree of coincidence between the feature quantity extracted by setting the same region in the non-object image and the template is calculated, and if the degree of coincidence is greater than Tp, it is determined as an error.

  The part information generation unit 223 selects the part candidate information 211 having the smallest size step from the part candidate information 211 determined to be valid by the identification rate determination unit 222 and sets it as the part information 212.

  In the present embodiment, the region candidate generation unit 220 is configured to sequentially expand the region. In this case, the region candidate information 211 that is first determined to be valid by the identification rate determination unit 222 becomes the region candidate information 211 with the smallest size. . That is, a compact part can be obtained while realizing the target value of the identification rate.

  In this regard, as already described, there is a relationship in which the identification rate basically increases as the site is larger. From this relationship, setting the target value of the identification rate is ideal as the size of the site. There is a limit size that does not exceed the target value when the identification rate exceeds the target value for a size that exactly realizes the target value, that is, a size that is larger than that. Actually, when the size of the part is discretely set as in the above-described embodiment, the ideal limit size cannot necessarily be set, but by setting the approximate limit size as described above, It is possible to improve the performance of highly accurate detection of an object in a partially concealed state by making the part compact.

  The method for determining the approximate limit size is not limited to the above-described embodiment. For example, a large number of sample frames based on random numbers are set between the upper limit and the lower limit of the size of the part, and each sample frame is set. A candidate for the region is generated, and after determining whether it is valid for all of these candidates, the minimum size of those that are validated is defined as the limit size, and all possible locals within the sample frame at each stage A different degree of approximation is conceivable, such as a configuration for searching for the order of adding regions.

  What approximate method is adopted can be determined in consideration of the purpose of the object detection device 1, for example. For example, the processing load on the learning device 2 basically increases as the accuracy of approximation increases. In this regard, the configuration can be selected as appropriate by comparing and considering the performance improvement and the load increase due to the compact part.

  Next, the operation of the learning device 2 will be described. FIG. 11 is a flowchart showing a schematic operation of the learning device 2. When the administrator turns on the power of the learning device 2 and operates the learning operation unit 20 to instruct the start of learning, the learning device 2 performs a learning process. Hereinafter, the learning process will be described with reference to FIG.

  The part candidate generation unit 220 of the learning control unit 22 sets a plurality of sample points in each object sample image (S20).

  When the feature point sampling method is employed, the part candidate generation unit 220 extracts feature points from each object specimen image, and extracts feature amounts at the respective feature points. Then, clustering focusing on the position and feature amount of the feature points between the object sample images is performed to generate feature point clusters, and the representative positions of the feature points belonging to each cluster are calculated as sample points. It should be noted that a cluster point having a small number of characteristic points to be attributed compared to the number of object sample images (for example, less than 60%) is not significant for detection of the object, and no sample point is set for the cluster.

  When the grid sampling method is employed, the part candidate generation unit 220 sets each of the equidistant lattice points set in advance in the object specimen image as a specimen point.

  When the random sampling method is adopted, the part candidate generation unit 220 generates a predetermined number of random points in each object specimen image based on the random number, and extracts a feature amount at each random point. Then, clustering focusing on the position and feature amount of the random points is performed between the object sample images to generate a cluster of random points, and the representative position of the random point belonging to each cluster is calculated as the sample point. As in the case of the feature point sampling method, sample points are not set for clusters with a small number of attributions.

  Next, the part candidate generation unit 220 sets a local region centered on each sample point (S21), and the learning unit 221 of the learning control unit 22 uses the object sample image and each non-object sample image in the local region. A feature amount is extracted (S22). The learning unit 221 temporarily stores, in the learning storage unit 21, sample information in which the extracted feature quantity is associated with the sample number of the sample image 210 and the coordinates of the sample point. By calculating and storing the feature amount at this stage and making it available at any time in later processing, it is possible to eliminate unnecessary duplication calculation.

  Subsequently, the part candidate generation unit 220 sequentially sets each local region as a reference local region (S23), and executes a loop process that repeats the processes of steps S24 to S31 for all the reference local regions. Note that the center of the reference local region is a part reference point.

  In this local region loop processing, the part candidate generation unit 220 first sets a minimum stage part (S24). That is, the part candidate generation unit 220 sets a minimum sample frame that surrounds five or more local regions other than the reference local region with the reference local region as the center. The part candidate generation unit 220 notifies the learning unit 221 of the local region included in the sample frame.

  The learning unit 221 reads the notified feature quantity of the local region from the sample information, and learns the identification criterion using the read feature quantity (S25). In this embodiment using a discriminator as a discrimination criterion, the learning unit 221 reads out feature quantities of the object specimen image and the non-object specimen image, and learns the discriminator by applying a boosting method to these feature quantities. To do.

  In another embodiment, other machine learning methods such as support vector machines are applied to learn the identification criteria.

  As another embodiment, a configuration in which a template is used as an identification criterion is also possible. In this case, the learning unit 221 reads out feature quantities of the object specimen image, averages these feature quantities for each local region, and creates a template. To learn.

  The learning unit 221 additionally stores the part candidate information 211 in which the learned identification reference is associated with the local region in the sample area in the learning storage unit 21, and the added part candidate information 211 is identified by the learning control unit 22. The determination unit 222 is notified.

  Upon receiving the notification of the addition of the part candidate information 211, the identification rate determination unit 222 reads the feature amount of the target specimen image and the non-target specimen image in the part indicated by the part candidate information 211 from the specimen information, and reads the feature. The identification rate is calculated by comparing the amount with the identification criterion indicated by the region candidate information 211 (S26).

  The discrimination rate determination unit 222 compares the calculated discrimination rate with the target value ε (S27). If the discrimination rate exceeds ε (“YES” in S27), the part candidate information 211 added in step S25. Is transferred to the part information 212, and a new part number is assigned (S28). In this embodiment in which the region is sequentially expanded, if the region candidate information 211 when the identification rate first exceeds the target value is the region information 212, the region information 212 is the minimum region information whose identification rate exceeds the target value ε. It becomes. When the part information 212 for the reference local area of interest is generated in this way, the part candidate generation unit 220 confirms whether all the local areas have been processed as the reference local area (S31), and has been processed. If so, learning ends ("YES" in S31), and if not completed ("NO" in S31), the remaining local area is processed (S23).

  On the other hand, if the identification rate does not exceed ε (“NO” in S27), region candidate generation unit 220 expands the region (S29). That is, the part candidate generation unit 220 sets a minimum sample frame that includes the reference local region and surrounds the local region by three more than the number of local regions included in the current sample frame. The site candidate generation unit 220 confirms whether or not the enlarged sample frame exceeds the preset upper limit (S30), and if it does not exceed the upper limit (“NO” in S30), the enlarged sample frame. The learning unit 221 is notified of the local region included in the learning unit 221, and the learning unit 221 learns the identification criterion using the notified feature amount of the local region (S 25).

  On the other hand, if the expanded sample frame exceeds the upper limit (“YES” in S30), the process proceeds to step S31 to determine the end of learning. That is, for the reference local region of interest, it has not been possible to generate site information with an identification rate exceeding the target value ε.

  After the learning process is completed, when the administrator operates the learning operation unit 20 to instruct the output of the part information 212, the learning control unit 22 reads the part information 212 from the learning storage unit 21 and outputs it to the learning output unit 23. Let

  In the above embodiment, the reference point B is set at the center of gravity of the sample image 210. However, the reference point B may be at an arbitrary position such as the upper left end or the lower right end of the sample image 210 as long as it is common among the feature points. .

  In many of the known machine learning methods including the boosting described above, it is common to end the learning either when the error rate is less than a set value or the number of iterations is equal to or greater than a preset number. It is. However, when the sample image used for learning is the same as the sample image used for identification rate calculation, the identification rate determination unit 222 sets a target value as a setting value for termination determination without calculating the error rate again, and the learning unit The error rate may be obtained by referring to the result of the end determination in 221.

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Learning apparatus, 10 Imaging part, 11 Image acquisition part, 12 Detection storage part, 13 Site information setting part, 14 Detection control part, 15 Detection output part, 20 Learning operation part, 21 Learning storage part, 22 learning control units, 23 learning output units, 30 object specimen images, 31 feature points (or random points), 32 regions, 33 sampling points, 120 part information, 121 part detection information, 140 detection magnification changing part, 141 part Detection unit, 142 Object determination unit, 143 Abnormality determination unit, 210 Sample image, 211 Site candidate information, 212 Site information, 220 Site candidate generation unit, 221 Learning unit, 222 Identification rate determination unit, 223 Site information generation unit

Claims (8)

A learning device that generates information on a part of an object used for detecting the object,
A specimen image storage unit that stores in advance the specimen image of the object and the specimen image of the non-object;
A site candidate generator that sets a predetermined site reference point in the sample image, and sequentially generates sites that include the site reference point and have different sizes;
A learning unit that learns an identification criterion for identifying the presence or absence of the part using an image feature of the part in at least the specimen image of the object for each part,
An identification rate determination unit that determines the identification rate by identifying the presence or absence of the site in each sample image according to the identification criterion of each site, and determines that the site where the identification rate exceeds a predetermined target value is valid,
A part information generation unit that generates part information including the part having the smallest size among the parts determined to be effective by the identification rate determination unit and the identification reference of the part;
A learning apparatus comprising: The learning device according to claim 1,
The part candidate generation unit sets a sample frame of the size surrounding the part reference point, and generates a partial region or a whole region in the sample frame as the part. The learning device according to claim 2,
The region candidate generation unit controls the size by setting a plurality of local regions that are analysis units of the image features in the sample image and increasing or decreasing the number of the local regions surrounded by the sample frame, A learning apparatus, characterized in that a collection of the local regions surrounded by a sample frame is generated as the part. The learning apparatus according to claim 3,
The site candidate generation unit subdivides the sample image at predetermined equal intervals to set the local region. The learning apparatus according to claim 3,
The part candidate generation unit extracts a plurality of feature points of the target object from a specimen image of the target object, and sets the local region at a position where each feature point is extracted. The learning apparatus according to claim 3,
The part candidate generation unit generates a random point in the sample image, and sets the local region at a position where the random point is generated. In the learning apparatus according to any one of claims 3 to 6,
The site candidate generation unit sets the site reference point at each position where the local region is set. An object detection device that detects the object imaged in an input image using information generated by the learning device according to any one of claims 1 to 7,
A part information storage unit for storing the part information generated by the part information generation unit;
Part detection that identifies the presence or absence of the part corresponding to the part information stored in the part information storage unit at each position of the input image, and outputs the position in the input image that is identified as having the part And
An object determination unit that detects the object when the position output by the part detection unit is concentrated beyond a preset object detection criterion;
An object detection apparatus comprising:

Images